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Abstract 

Previous  research  concerning  aircraft  maintenance  officer  training  focused  on  the 
ctxitent  of  the  training  curriculum  of  the  Aircraft  Maintenance  and  Munitions  Officers 
Course  (AMMOC).  Conversely,  this  study  sought  improvement  in  aircraft  maintenance 
officer  training  by  evaluating  the  guidance  and  support  provided  to  AMMOC.  Two 
methods  were  employed  for  identifying  such  oppOTtunities.  First,  a  descriptive  model  of 
the  training  system  supporting  AMMOC  was  developed  and  analyzed.  Second,  feedback 
was  solicited  from  AMMOC  instructors  through  the  use  of  a  two-round  Delphi.  The 
Delphi  was  employed  to  develop  a  consensus  among  the  instructors  regarding  what 
improvement  opportunities  existed  and  potential  means  for  taking  advantage  of  these 
oppOTtunities.  Findings  indicate  that  AMMOC  may  be  improved  by:  facilitating  better 
communication  between  AMMOC,  its  customers,  and  other  organizations;  empowering 
AMMOC  instructors  with  more  control  over  the  course  training  standard  (CTS),  student 
scheduling,  and  customer  feedback;  and  providing  instractors  more  time  for  course 
development  by  assessing  and  satisfying  AMMOC’ s  manpower  requirements  and  refining 
training  development  and  manning  policies. 
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QUALITY  IMPROVEMENT  ANALYSIS  OF  THE  AIR  FORCE 
AIRCRAFT  MAINTENANCE  AND  MUNITIONS 
OFFICERS  COURSE 


I.  Introduction 


Chapter  Overview 

In  recognition  of  the  importance  of  effective  and  efficient  training,  this  thesis 
presents  a  quality  improvement  analysis  of  training  guidance  and  support  for  the  Aircraft 
Maintenance  and  Munitions  Officers  Course  (AMMOC).  This  chapter  justifies  the 
analysis  by  presenting  the  general  issues  surrounding  training  such  as  the  general  status  of 
today's  Air  Force  and  aircraft  maintenance  officer  training.  The  chapter  then  provides  the 
rationale  for  conducting  a  quality  improvement  analysis  of  AMMOC.  The  resulting 
research  objective  and  investigative  questions  follow.  Finally,  we  provide  a  summary  of 
the  methodology  employed  with  a  description  of  its  scope  and  limitations. 

General  Issue 

According  to  General  Viccellio,  Commander  and  Chief  of  Air  Education  and 
Training  Command  (AETQ,  "the  decade  of  the  nineties  has  brought  about  significant 
changes  in  the  way  we,  the  Air  Force,  accomplish  our  mission"  (1993).  Since  the  mid 
1980s,  Air  Force  end  strength  has  plummeted  to  its  lowest  level  in  more  than  40  years 
(Perini,  1992:  39).  As  a  result  of  smaller  budgets,  the  Air  Force  has  undergone  extensive 
reductions  in  force,  base  closures,  and  deactivation  of  major  weapons  systems  (Perini, 
1992:  39).  In  the  midst  of  this  reduction  in  available  resources,  the  Air  Force  has 
wimessed  the  demise  of  the  former  Soviet  Union  and  the  emergence  of  other  national 
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security  threats  in  the  Middle  East,  Eastern  Europe,  and  Southeast  Asia.  A  smaller  Air 
Force  in  the  presence  of  this  unstable  world  environment  underscores  the  need  for  flexible 
and  responsive  training  systems  to  achieve  the  Air  Force  vision  of  "Global  Reach  -  Global 
Power." 

Aircraft  Maintenance  Officer  Training.  The  Air  Force  designated  1992  as  the 
"Year  of  Training."  During  this  "Year  of  Training,"  Air  Training  Command  (ATC) 
conducted  a  "top-to-bottom"  review  of  how  the  Air  Force  trains  its  people.  Several  issues 
prompted  this  intensive  review  of  Air  Force  training.  They  included  the  reality  of  a 
smaller  Air  Force,  the  importance  of  training  in  maintaining  a  combat  capability,  and  the 
Air  Force  leaderships'  strong  emphasis  on  creating  a  Quality  Air  Force  (QAF)  (Boles, 
1994).  AcccHding  to  General  Viccellio,  the  questions  asked  during  ATCs  review  were: 
"How  do  we  conduct  training?  Do  we  do  it  smartly?  Who  does  it?  What's  the  content? 
Do  we  do  it  effectively  and  efficiently?  Do  we  do  it  in  standardized  way?"  (Graham, 

1993:  38). 

ATCs  review  of  training  was  broad  in  scope;  it  encompassed  flying  training, 
technical  training,  and  education  (Viccellio,  1993).  However,  it  is  unclear  whether  this 
review  specifically  scrutinized  the  policies  and  practices  pertaining  to  training  of  Aircraft 
Maintenance  Officers  (AMOs).  According  to  General  Viccellio,  logistics  officer  training 
has  virtually  gone  unchanged  during  the  past  twenty  years  and  deserves  a  "hard  look" 
(1993).  Because  of  AETC's  recent  emphasis  on  quality  and  its  apparent  willingness  to 
change,  we  believe  this  is  an  ideal  opportunity  to  exantine  the  training  of  an  important 
component  of  logistics  officer  training-the  Aircraft  Maintenance  and  Munitions  Officers 
Course  (AMMOC). 

The  Need  for  Scrutiny.  There  have  been  several  studies  addressing  AMO 
traiiting  over  the  years.  The  most  recent  study  addressing  AMO  training  is  a  1991 
Training  Evaluation  of  AMMOC.  This  evaluation  focused  on  how  well  AMM(X; 
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prepared  its  graduates  for  future  jobs.  The  study  reported  that  22  percent  of  "recent 
graduates"  rated  AMMOC  as  either  marginal  or  unsatisfactory  in  preparing  maintenance 
officers  for  their  jobs  (Department  of  the  Air  Force,  1991a:  2).  Despite  this  high  number 
of  marginal  and  unsatisfaaory  ratings,  the  training  evaluation  report  concluded  that 
AMMOC  was  "providing  the  field  with  qualified,  performance-ready  graduates...” 
(Department  of  the  Air  Force,  1991a:  4). 

Other  than  the  1991  Training  Evaluation  Report,  the  most  recent  studies  were 
conducted  by  Bair  and  Gatewood  (1982),  and  Frisbee  (1988).  Both  of  these  studies 
primarily  focused  on  an  age-old  debate  over  the  "technical  versus  managerial"  orientations 
of  aircraft  maintenance  officers.  As  part  of  their  research,  both  studies  include  a 
significant  discussion  on  AMMOC.  Bair  and  Gatewood  assert  that  "while  AMMOC 
graduates  considered  their  preparation  satisfactory,  they  feel  it  is  too  theoretical  and 
quickly  loses  relevance"  (1982: 77).  Though  Frisbee  does  not  explicitly  investigate  the 
appropriateness  of  AMMOC's  curriculum,  he  does  suggest  that  the  training  AMMOC 
provided  may  have  been  inadequate  (1988: 2).  Although  evidence  suggests  that  problems 
with  AMMOC  curriculum  might  exist,  it  is  unclear  what  the  causes  of  such  problems 
might  be. 

A  New  Approach.  While  prior  studies  have  expressed  some  dissatisfaction  with 
AMMOC,  none  has  addressed  AMMOC  in  any  other  context  than  maintenance  managers' 
satisfaction  with  it  If  a  significant  dissatisfaction  with  AMMOC  has  existed  over  the  past 
15  years  as  suggested  by  prior  studies,  a  deeper  focus  than  curriculum  content  is  in  order. 
Scrutiny  must  not  be  limited  to  AMMOCs  product  but  must  consider  the  policies, 
processes,  and  people  who  comprise  the  "system"  of  training  and  development 
Improvements  in  the  policies  and  processes  associated  with  training  development  and 
operations  may  hold  the  key  to  an  improved  product  and  higher  customer  satisfaction. 
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An  evaluation  of  organizational  policies  and  processes  requires  a  broader 
perspective  than  what  was  achieved  through  related  studies  of  AMO  training.  The 
quantitative  methodologies  of  previous  studies  completely  ignored  the  processes  and 
concepmal  aspects  of  the  organizations  which  develop  and  operate  training  systems. 
Bryman  (1989)  points  out  that  quantitative  methodologies  tend  to  evaluate  organizations 
fixMn  the  perspective  of  "abstract  variables"  preconceived  by  the  researchers.  Conversely, 
qualitative  methodologies  allow  researchers  to  evaluaffi  circumstances  in  the  context  of 
the  organizations  in  which  they  exist,  from  the  perspective  of  members  of  the  organization 
through  the  use  of  methods  such  as  document  reviews,  unstructured  interviews,  and  other 
techniques  that  directly  involve  the  stakeholders  of  the  system.  Qualitative  research  can 
provide  a  perspective  that  many  quantitative  methodologies  cannot  (Bryman,  1989:  139- 
141).  Such  methodologies  that  focus  on  organizational  processes  and  individuals  are  also 
consistent  with  Air  Force  leadership's  recent  emphasis  on  the  management  philosophies  of 
(Quality  Air  Force. 

Quality  Air  Force.  Quality  Air  Force  (QAF)  is  a  management  philosophy  which 
seeks  continuous  improvements  in  the  way  the  Air  Force  accomplishes  its  mission.  Unlike 
traditional  management  concepts,  QAF  disavows  centralized  control  of  day  to  day 
operations  and  processes  that  take  place  at  the  working  level.  Rather,  QAF  encourages 
empowerment  of  individuals  at  "the  point  of  contact,"  and  a  "customer  focus" 
(Department  of  the  Air  Force,  1993f:  1-9, 11-6). 

The  QAF  concept  of  "customer  focus"  recognizes  that  customers'  needs  are  the 
organization's  sole  reason  for  existence.  Even  though  all  organizational  activities  do  not 
interact  directly  with  the  customer,  ultimately  each  activity  should  contribute  somehow  to 
the  organization's  service  or  product.  Karl  Albrecht,  author  of  Service  within,  places 
every  organizational  activity  in  perspective  with  his  assertion  that  "if  you're  not  serving  the 
customer,  your  job  is  to  serve  someone  who  is"  (1990:  3). 
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While  AMMOC's  external  customers  are  the  ones  who  should  ultimately  be 
satisfied,  a  focus  on  AMMOC  as  an  internal  customer  of  the  training  system  is  of  vital 
importance.  According  to  Osborne  and  Gaebler,  "customers  are  the  most  important 
people  for  an  organization...and  management  is  there  to  serve  those  who  serve  customers" 
(1992: 172).  Within  this  framework,  AMMOC  instructors  are  the  providers  of  service 
(AMO  training),  and  the  training  system  under  which  AMMOC  operates  embodies  the 
management  who  serves  the  AMMCK!  instructors  with  guidance  and  support  We  define 
guidance  and  support  as  policy,  resources,  services,  or  anything  else  that  affects 
AMMOC's  ability  to  accomplish  its  mission.  The  QAF  concept  of  "customer  focus"  insists 
that  AMMOC,  as  customers,  play  an  important  role  in  defining  the  service  its  management 
provides. 

Another  closely  related  QAF  concept  is  "empowerment  at  the  point  of  contact." 
Empowerment  means  that  individuals  who  are  tasked  with  accomplishing  the  mission,  or 
"serving  a  customer,"  are  provided  the  resources  and  authority  to  actually  succeed. 

Worker  involvement  in  improving  internal  processes  is  regarded  as  an  important  aspect  of 
empowerment.  In  her  quality  article  series,  Keane  quotes  General  McPeak,  Air  Force 
Chief  of  Staff,  as  saying,  "It's  often  the  person  who's  right  in  contact  with  the  work  who 
understands  what  needs  to  be  fixed  about  that  process"  (1992: 7).  Recognizing  that 
"empowerment  at  the  point  of  contact"  is  a  basic  principle  of  QAF,  and  that  AMMOC  is 
the  "point  of  contact"  in  the  training  of  AMOs,  it  must  be  true  that  AMMOC  instructors 
are  individuals  the  Air  Force  wants  to  empower. 

Research  Objective 

Within  the  framework  of  the  Quality  Air  Force,  the  objective  of  this  thesis  research 
is  to  identify  opportunides  to  improve  the  suppon  and  guidance  provided  to  the  Aircraft 
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Maintenance  and  Munitions  Officers  Course  (AMMOC).  To  that  end,  this  research 
addresses  the  following  investigative  questions: 

1.  What  areas  related  to  the  guidance  and  support  provided  to  AMMOC  are  the 
strongest  candidates  for  improvement  initiatives? 

2.  How  can  the  identified  areas  be  improved? 

Conduct  of  Study 

This  research  seeks  to  achieve  the  research  objective  above  through  a  combination 
of  two  means.  The  first  is  to  develop  a  narrative  description  or  model  of  the  training 
system  to  which  AMMOC  belongs.  This  training  system  description  is  derived  from 
reviews  of  Air  Force  publications  and  unstructured  discussions  with  training  practitioners. 
The  training  system  is  then  evaluated  by  the  researchers  to  identify  potential  improvements 
in  OTganizatitmal  structure.  The  training  system  description  is  presented  in  Chapter  n,  and 
discussed  in  Chapters  IV  and  V. 

The  second  method  for  achieving  our  research  objective  is  to  obtain  the  opinions 
of  AMMOC  instructors  regarding  how  training  guidance  and  policy  may  be  improved.  As 
“customers”  of  training  policy  and  guidance,  AMMOC  instructors  are  in  an  excellent 
position  to  identify  opportunities  to  improve  the  training  support  they  are  afforded.  Many 
of  the  AMMOC  instructors  are  not  only  internal  customers  of  the  training  system,  but  are 
also  experienced  as  external  customers  of  AMMOC,  as  former  AMMOC  students,  and  as 
former  supervisors  of  AMMOC  graduates.  Furtheimore,  because  AMMOC  employs  its 
former  students  as  instructors,  in  a  sense,  AMMOC  is  its  own  customer. 

The  methodology  we  chose  for  acquiring  the  group  opinion  among  AMMOC 
instructors  is  known  as  the  Delphi  technique.  We  chose  this  technique  because  it 
facilitates  the  free  expression  of  ideas  among  the  AMMOC  staff  without  the  stigma  of  any 
particular  instructor  attached  to  them.  This  Delphi  procedure  consisted  of  two  rounds  of 
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a  question  and  feedback  process.  The  first  round  solicited  a  diversity  of  ideas  from 
AMMOC  instructors  through  open-ended  questions  regarding  the  support  and  guidance 
AMMOC  is  provided  and  how  it  may  be  improved.  The  second  round  anonymously 
presented  each  AMMOC  instructor's  ideas  to  each  of  the  other  instructors  for  a  vote  to 
determine  if  ccmsensus  existed  among  the  instructors.  Further  discussion  and  analysis  of 
the  Delphi  is  presented  in  Chapters  m  and  IV,  respectively.  The  conclusions  and 
recommendations  of  this  thesis  are  based  on  both  of  the  methodologies  discussed  above 
and  are  presented  in  Chapter  V. 

Scope.  The  scope  of  this  research  is  limited  to  identifying  opportunities  for 
improving  the  support  and  guidance  provided  to  the  Aircraft  Maintenance  and  Munitions 
Officers  Course  (AMMOC).  Sources  for  identifying  opportunities  are  limited  to  AMMOC 
instractors  and  the  researchers.  The  support  and  guidance  under  consideration  may 
originate  from  any  organization,  at  any  level. 

Limitations- 

1.  During  the  conduct  of  this  research  effort,  AETC  was  in  the  midst  of 
reviewing  and  redefining  technical  training  policy.  While  every  attempt  has  been  made  to 
incorporate  the  most  recent  policy  into  this  research,  it  is  possible  that  some  of  the  policies 
relevant  to  this  thesis  have  changed. 

2.  The  respondents  for  the  Delphi  procedure  were  limited  to  AMMOC 
instructors.  This  is  because  of  the  importance  of  "customer  focus"  in  process 
improvement,  and  because  AMMOC  is  a  primary  customer  of  the  training  support  it  is 
afforded.  We  recognize  that  other  credible  sources  of  training  expertise  were  available, 
but  not  included  in  this  study. 
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Summary 

This  chapter  established  the  focus  of  this  research  effort.  The  unstable  world 
environment  and  shrinking  military  budget  and  force  structure  have  elevated  the  need  for 
effective  and  efficient  training.  The  means  for  obtaining  the  best  training  possible  can  be 
found  in  the  concepts  embodied  in  the  Quality  Air  Force  initiative.  The  Air  Force's 
aircraft  maintenance  officer  training  system  must  continuously  improve  its  processes,  it 
must  assume  both  an  internal  and  external  customer  focus,  and  it  must  empower  at  the 
point  of  contact.  This  thesis  proposes  a  research  objective  and  methodology  which  meet 
these  requirements. 

The  chapters  that  follow  describe  the  current  training  system  and  relevant 
processes,  the  methodology  employed  to  achieve  the  research  objective,  the  results  and 
findings  of  the  research,  and  finally,  the  conclusions  and  recommendations  derived  from 
the  findings. 
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Chapter  Overview 

Before  pursuing  the  research  objective  of  identifying  opportunities  to  improve  the 
guidance  and  support  provided  to  AMMOC,  relev  ant  informadon  concerning  AMMOC 
and  its  supporting  system  should  be  considered.  This  chapter  provides  an  overview  of  the 
AMO  career  field  and  training,  relevant  research,  instructional  systems  development,  and  a 
description  of  the  global  training  system  in  which  AMMOC  exists. 

The  Aircraft  Maintenance  Officer 

AMOs  are  responsible  for  managing  and  leading  large  organizations  tasked  with 
maintaining  the  readiness  of  multimillion  dollar  aircraft  and  equipment,  and  supporting 
flying  operations  during  peacetime  and  war.  Aircraft  maintenance  is  an  extremely  broad 
discipline  which  requires  technical  expertise,  managerial  competence,  and  leadership 
ability  (Department  of  the  Air  Force,  1991b:  All  -  A16).  Duty  assignments  for  AMOs 
are  possible  at  operational  wings,  air  logistics  centers,  and  major  command  headquarters 
throughout  the  world  in  support  of  various  weapons  systems  and  missions  (Department  of 
the  Air  Force,  1991c;  92-97).  According  to  AFR  36-1,  Officer  classification,  aircraft 
maintenance  and  munitions: 

encompasses  the  functions  of  program  fcnmulation,  policy  planning,  production 
management,  quality  control,  inspection,  and  direction  of  aircraft  maintenance, 
avicMiics,  and  munitions  activities...[the  aircraft  maintenance  officer]  manages  all 
levels  of  aircraft  maintenance,  its  associated  subsystems,  and  munitions 
activities...organizes  units  to  accomplish  maintenance...allocates  available 
maintenance  resources  to  ensure  maximum  equipment  readiness... assigns  work, 
establishes  priorities,  and  controls  or  monitors  production...Implements  aircraft 
emergency  war  order  and  combat  turn  activities...confers  with  operations  activities 
to  determine  operational  problems  and  support  requirements... observes  work  in 
progress  and  reviews  actions  for  quality  and  compliance  with  technical,  safety,  and 
security  directives.,  .evaluates  effectiveness  of  systems  operation  and  recommends 


changes  in  operational  use,  equipment  modification,  and  maintenance 
procedures... directs  nuclear  receipt,  storage,...maintenance,...safety,  and  security 
activities....  (Department  of  the  Air  Force,  1991b:  All  -  A16) 

Training  for  entry-level  AMOs  consists  of  a  mix  of  formal  technical  training  and 
on-the-job  training.  Completion  of  AMMOC  and  a  minimum  of  six  months  of  experience 
in  an  AMO  duty  position  are  required  for  designation  as  a  fully-qualified  AMO.  Although 
not  required,  the  Air  Force  prefers  AMOs  with  undergraduate  degrees  in  engineering, 
management,  or  the  physical  sciences  (Department  of  the  Air  Force,  1991b:  11-15-  11- 
16). 

History  of  the  AMO  Career  Reid  and  Training 

Formal  training  of  AMOs  seems  to  have  become  a  sacred  institution.  The  United 
States  Army  first  established  a  need  for  aircraft  maintenance  capabilities  in  1907  when  its 
Aeronautical  Division  possessed  a  single  aircraft.  Army  officers  learned  aircraft 
maintenance  skills  on  the  job  until  1917  when  formal  aircraft  maintenance  training  was 
established  by  the  Air  Corps  Technical  School  at  Kelly  Field,  Texas.  In  1921,  the  course 
was  renamed  the  Air  Service  Mecharucs  School  and  relocated  to  Chanute  Field,  Illinois, 
where  it  eventually  evolved  into  the  Aircraft  Maintenance  and  Munitions  Officers  Course 
(AMMOC)  (Chandler  and  Fox,  1988:  3).  In  1993,  after  nearly  76  years  of  aircraft 
maintenance  training,  AMMOC  was  relocated  to  Sheppard  AFB,  Texas,  during  the 
closure  of  Chanute  AFB. 

Throughout  the  years,  the  AMO  career  field  and  its  associa^  training  have 
undergone  a  cycle  of  change.  This  cycle  began  with  divergence  of  the  munitions 
maintenance  and  aircraft  maintenance  activities  in  1927  and  the  subsequent  transfer  of 
munitions  training  from  Chanute  to  Lowry  Field  at  Denver,  Colorado,  in  1938.  Higher 
degrees  of  specialization  continued,  and  by  1955,  the  aircraft  and  armament  career  fields 
had  been  shredded  into  sixteen  officer  specialties.  In  1963,  justified  by  the  increasing 
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complexity  of  aviation  and  annament  technology,  the  avionics/munitions  specialty  was 
created. 

This  trend  of  specializing  aircraft  maintenance  disciplines  reversed  in  the  1970s. 

By  1976,  the  avionics  and  aircraft  maintenance  specialties  were  recombined,  and  during 
the  1980s,  after  years  of  debate,  the  munitions  and  aircraft  maintenance  career  fields  were 
also  reconsolidated  (Qtandler  and  Fox,  1988:  7-19).  Today,  the  aircraft,  a  /ionics,  and 
munitions  maintenance  disciplines  comprise  a  single  Air  Force  officer  career  field 
(Department  of  the  Air  Force,  1991b:  All- 18),  and  all  maintenance  officer  technical 
training  is  conducted  by  AMMOC  at  Sheppard  AFB,  Texas  (Lawlor.  1994). 

AMMOC 

AMMCK:  is  a  formal,  entry-level,  technical  training  course  conducted  by  the  362nd 
Training  Squadron  at  Sheppard  AFB,  Texas.  Depending  on  the  background  of  the 
students,  they  will  attend  at  least  one  of  four  aircraft  maintenance  and  munitions  officers 
courses.  The  majority  of  students  are  junior  officers  who  are  new  to  the  aircraft 
maintenance  officer  career  field.  These  students  attend  the  "basic"  course  which  is 
presently  18  weeks  in  length.  Shoner  courses  exist  for  Air  National  Guard  and  Air  Force 
Reserve  officers  with  aircraft  maintenance  experience,  and  for  rated  officers  who  are 
cross-training  into  the  maintenance  career  field.  A  fourth  course  AMMOC  offers  is  a 
follow-on  to  the  basic  course  and  focuses  on  nuclear  weapons  theory,  policy,  and 
procedures  (Department  of  the  Air  Force,  1993h:  3-64).  Thioughout  this  thesis,  the  term 
"AMMOC"  refers  to  the  basic  course  described  above. 

Course  Content.  AMMOC  provides  an  overview  of  a  wide  variety  of  subjects 
such  as  aircraft  and  munitions  systems  theories,  maintenance  management  policies,  and 
maintenance  management  information  systems.  The  course  mostly  consists  of  classroom 
lecture  and  discussion,  but  also  inclucfes  tours  of  aircraft  maintenance  activities  and  a  few 
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hands-on  maintenance  tasks.  Furthermore,  Block  DC  of  the  course  is  a  simulation  of  an 
actual  maintenance  operation.  The  proficiencies  required  for  completion  of  AMMOC  are 
extremely  basic.  Learning  objectives  typically  call  for  students  to  identify  facts  or 
principles  concerning  various  subjeas.  Table  2-1  below  illustrates  the  general  flow  of  the 
major  subject  areas  throughout  the  course  (Department  of  the  Air  Force,  1993a). 


Table  2-1.  Sequence  of  AMMOC  Training  (Department  of  the  Air  Force,  1993a) 


Block 

Description 

I  (9.5  days) 

1 

Orientation,  Maintenance  Staff  Agencies, 
Production  Activities 

n  (10  days) 

Microcomputers,  Personnel, 
Communications  Security,  Operations 
Security 

in  (10  days) 

Munitions,  Maintenance  Storage  and 
Handling,  Explosive  Safety 

rv  (10  days) 

Aerodynamics,  Structures,  Nuclear 
Weapons,  Preventative  Maintenance, 
Reliability  and  Maintainability 

V  (10  days) 

Electricity,  Aircraft  Power  Supply  System, 
Basic  Avionics  System  and  Associated 
Equipment, 

VI  (10  days) 

Jet  Engines,  Aircraft  Systems,  Suppon 
Equipment 

Vn  (10  days) 

Supply,  Resource  Management,  Munitions 
Accountability 

Vin  (10  days) 

Maintenance  Forms,  Exceptional  Releases 
and  Maintenance  Management  Information 
Systems 

IX  (8  days) 

Production  Planning  and  Maintenance 
Planning 

iNote:  Only  a  representative  selection  oi 

f  course  topics  is  included  in  this  table) 

Course  Operation.  As  depicted  in  the  table  above,  AMMOC  is  currently  an 


18- week  course  which  consists  of  nine  two-week  blocks  of  instruction.  Usually, 
AMMOC  receives  a  new  class  every  two  weeks.  Students  begin  and  progress  through 


AMMOC  in  groups  of  approximately  15.  AMMOC’s  training  day  is  designed  to  include 
eight  50-minute  sessions  of  instruction,  a  lunch  period,  and  10  minute  breaks  throughout 
the  day.  Generally,  each  block  of  instruction  is  administered  by  a  single  instructor.  Once 
a  block  of  instruction  is  completed,  students  progress  to  the  next  block  and  instructors 
receive  new  groups  of  students.  The  AMMOC  instructor  staff  is  comprised  of  16  AMOs 
and  civilian  instructors  with  varying  levels  of  aircraft  maintenance  experience  (Dalton, 
1994). 

It  should  be  noted  that  during  the  1994  AMO  Utilization  and  Training  Workshop 
(U&TW),  it  was  agreed  that  AMMOC  should  be  streamlined  by  eliminating  some 
elements  of  curriculum  which  are  already  provided  by  other  sources  of  training.  It  was 
also  decided  that  the  course's  length  should  be  commensurately  reduced  from  18  weeks  to 
approximately  13  weeks.  The  revised  curriculum  is  planned  to  become  effective  in  July 
1995  (Broaidt,  1994). 

Related  Research 

A  thorough  review  of  literature  and  informal  discussions  with  the  researchers' 
colleagues  reveal  only  a  few  prior  research  efforts  that  are  relevant  to  this  thesis-one 
informal  and  three  formal.  The  most  relevant  forum  was  an  informal  quality  improvement 
workshop  conducted  by  AMMOC  in  1992.  The  three  formal  studies  were  a  1991 
Training  Evaluation  Report  (TER)  on  AMMOC,  and  two  AFIT  theses. 

The  purpose  of  AMMOC's  1992  quality  improvement  workshop  was  to  define 
AMMOC's  "mission"  and  "vision,"  and  to  identify  and  resolve  problems  that  threatened 
either  of  them.  The  participants  in  this  workshop  were  AMMOC  instructors,  including 
one  of  the  authors,  and  course  development  personnel  at  Chanute  AFB,  Illinois.  The 
participants  defined  AMMOC's  mission  and  vision  as  the  following: 
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Mission:  Preparing  officers  to  lead  and  manage  in  an  aircraft  maintenance  and 
munitions  environment.  (AMMOC,  1992) 

Vision: 

[1.]  Be  on  the  leading  edge  and  recognized  as  the  authority  in  the  aircraft 
maintenance  and  munitions  management  field, 

[2.]  the  'place  to  be'  for  both  instractors  and  students, 

[3.]  fielding  a  course  which  is  both  challenging  and  rewarding,  making  students 
proud  to  be  an  AMMOC  graduate.  (AMMOC,  1992) 

During  this  workshop,  participants,  through  the  use  of  an  experienced  facilitator, 
identified  problems  affecting  AMMOC.  On  an  anonymous  basis,  the  workshop 
participants  ranked  the  severity  and  solvability  of  each  problem.  The  participants  based 
the  solvability  of  each  problem  on  AMMOC's  capability  and  authority  to  implement 
corrective  actions.  The  aggregate  of  solvability  and  severity  rankings  established  priorities 
for  future  management  attention.  The  general  problem  areas  identified  were  student 
measurement,  course  content,  course  operations,  student  administration,  reference 
materials,  morale,  funding,  centralized  control,  equipment,  and  facilities. 

This  workshop's  emphasis  on  solvability  in  establishing  priorities  for  problem  areas 
indicates  that  the  workshop  was  primarily  focused  on  solving  AMMOC's  internal 
problems.  Conversely,  this  thesis  is  primarily  intended  to  address  issues  beyond 
AMMOC's  control.  Problems  addressed  by  this  workshop  which  the  researchers  found 
through  subsequent  research  to  potentially  be  beyond  AMMOC's  control  were  the  basis  of 
the  first  round  Delphi  questionnaire  described  in  Chapter  in  and  presented  in  Appendix  E. 

The  most  recent  evaluation  of  AMMOC  was  conducted  by  the  333()th  Technical 
Training  Wing  at  Chanute  AFB,  Illinois,  in  1991.  The  purpose  of  the  evaluation  was  to 
determine  if  AMMOC  graduates  were  using  their  training,  and  how  well  the  training 
prepared  the  graduates  for  duty.  The  evaluation  was  based  on  internal  performance 
measures,  inspection  results,  interviews  of  two  AMMOC  graduates,  and  questionnaires 
administered  to  106  AMMOC  graduates.  As  shown  in  Table  2-2, 78  percent  of  the 
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graduates  rated  AMMOC  as  satisfactory  or  higher  in  preparing  AMOs  for  duty;  22 
percent  of  the  graduates  rated  AMMOC  otherwise  (IDepaitment  of  the  Air  Force, 
1991a:  2). 


Table  2-2.  AMMOC  Graduate  Ratings  of  AMMOC  Perfomiance 
(Department  of  the  Air  Force,  1991a:  2) 


Ratings 

Number 

Percentage 

Outstanding 

4 

4 

Excellent 

19 

20 

Satisfactory 

51 

54 

20 

20 

Unsatisfactory 

2 

2 

Total 

96 

100 

This  Training  Evaluation  Report  was  the  first  evaluation  of  AMMOC  since 
maintenance  and  nuclear  munitions  training  had  been  consolidated  into  a  single  course  in 
1990  (Haynes,  1994).  The  fact  that  the  course  was  undergoing  instructional  systems 
development  during  the  period  of  the  evaluation  was  offered  as  a  possible  reason  for  some 
low  ratings  of  AMMOC's  performance  by  these  graduates.  The  report  also  acknowledged 
deficiencies  in  the  Core  Automated  Maintenance  System  (CAMS)  training.  All  other 
aspects  of  the  report  were  positive  and  the  course  was  ultimately  hailed  as  one  that 
provided  "performance-ready  graduates..."  (Department  of  the  Air  Force,  1991a:  2). 

Bair  and  Gatewood's  (1982)  AFTT  thesis  examined  the  “managerial  versus 
technical”  profiles  of  AMO  jobs,  the  relevance  of  AMMOC  curriculum,  and  the  relevance 
of  the  Air  Force’s  career  development  philosophies  for  AMOs.  Surveys  of  AMMOC 
graduates  indicated  that  AMOs  regarded  AMMOC’s  insmiction  on  technical  subjects  as 
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“slightly  useful,”  and  AMMOC’s  instruction  on  management  oriented  subjects  as  “useful.” 
The  surveys  also  indicated  that  AMOs  thought  AMMCX^  was  “too  theoretical”  (Bair  and 
Gatewood,  1982: 76-77). 

In  the  most  recent  AFTT  thesis  pertaining  to  AMO  training,  Frisbee  (1988)  focused 
on  the  “technical  versus  admiitistrative”  orientations  of  AMOs.  In  this  case,  Frisbee 
surveyed  maintenance  personnel  in  the  Strategic  Air  Command  (SAC)  to  determine  their 
preferences  for  a  “techrucally”  oriented  AMO  versus  an  “admirtistratively”  oriented  AMO. 
Frisbee  found  that  maintenance  personnel  desired  AMOs  who  were  equally  oriented  with 
administrative  and  technical  skills  (1988: 76-77).  One  apparent  premise  of  Frisbee’s  study 
was  that  trairung  provided  to  AMOs  should  reflect  the  orientations  for  AMOs  needed  by 
the  gairung  organizations. 

Both  of  the  above  theses  were  seemingly  pursued  under  the  premise  of  a 
dissatisfaction  with  AMO  trairung.  This  is  apparent  fircnn  both  studies’  reference  to  a 
general  officer’s  assertion  that  AMMOC  was  incapable  of  providing  students  with  the 
needed  skills  (Bair  and  Gatewood,  1982: 1 1-12;  Frisbee,  1988:  2-3).  The  TER  and  both 
AFTT  theses  employed  methodologies  that  were  predominantly  quantitative,  and  tended  to 
focus  on  either  training  curriculum  or  customer  opinion.  Conversely,  through  the  use  of  a 
qualitative  methodology,  this  thesis  focuses  on  "training  system"  processes  which 
ultimately  lead  to  training  curriculum  and  customer  opinion. 

Training  and  instructionai  Systems  Deveiopment 

The  Department  of  Defense  defines  training  as  "instruction  and  applied  exercises 
for  the  acquisition  and  retention  of  skills,  knowledge,  and  attitudes  required  to  accomplish 
military  tasks"  (Department  of  Defense,  1987:  1).  Likewise,  the  purpose  of  Air  Force 
technical  training  is  to  impart  the  knowledge  and  skills  required  to  qualify  for  an  Air  Force 
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Specialty  (AFS).  AMMOC  is  classified  as  "resident  regular"  technical  training  which 
denotes  AFS  initial  or  advancement  training  (Department  of  the  Air  Force,  1994c:  5-8). 

Instmctional  Systems  Development.  The  Air  Force  adopted  Instructional 
Systems  Development  (ISD)  as  its  method  for  developing  training  in  1965.  According  to 
Air  Force  Handbook  36-2235,  Irrformationfor  designers  of  instructional  systems,  ISD  is 
a  "conceptual  adaptation  of  the  systems  engineering  process  to  the  problems  of 
developing,  implementing,  and  evaluating  instruction"  (Department  of  the  Air  Force, 
1993c:  2-3)  and  is  intended  to  develop  relevant  instruction  that  can  impart  the  needed 
"skill,  knowledge,  and  attitude"  to  satisfy  job  performance  (1993c:  2-3).  ISD  is  based  on 
the  following  key  concepts. 


[1]  The  instraction...is  determined  by  job  requirements. 

[2]  Instruction  requirements  are  based  on  a  behavioral  analysis  that  results  in 
measurable,  observable  objectives. 

[3]  Instruction  is  provided  only  on  that  portion  of  the  job  requirements  that  the 
student  has  not  already  mastered. 

[4]  Measurement  is  keyed  to  the  objectives,  and  students  are  measured  against 
criteria  rather  than  in  comparison  to  a  norm. 

[5]  The  instruction  is  student  oriented.... 

[6]  The  student  and  the  instructor  both  know  the  instructional  goals  and  when 
they  have  been  attained. 

[7]  If  the  system  doesn't  teach,  it  may  be  the  fault  of  the  system  design.... 
(Department  of  Air  Force,  1993c:  13-14) 

Shown  in  Figure  is  the  Air  Force's  recently  revised  model  of  ISD.  This  ISD 
noodel  portrays  the  processes  and  functions  associated  with  training  development  An 
important  characteristic  of  the  new  ISD  model  is  that  it  recognizes  that  processes  of 
instructional  systems  development  transcend  the  organizational  functions  of  management 
support,  administration,  and  prcxluct  delivery.  Another  significant  modification  is  the  ISD 
model's  emphasis  on  Quality  Air  Force  (QAF)  philosophies  such  as  customer  focus, 
teamwork,  empowerment,  and  continuous  evaluation. 
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The  ISD  model  presented  in  Figure  2-1  is  only  a  conceptual  model;  it  does  not 
precisely  illustrate  the  activities  involved  in  training  development.  Each  of  the  processes 
depicted  in  Figure  2-1  can  involve  a  multitude  of  activities.  The  precise  activities  required 
for  development  of  a  particular  course  depend  on  factors  such  as  training  objectives, 
ctxnplexity  of  subject  matter,  available  time,  resources,  and  number  of  students  who 
require  training  (Department  of  the  Air  Force,  1993c:  13).  The  following  paragraphs 
explain  the  general  nature  of  the  processes  involved  in  ISD. 


Figure  2-1.  ISD  Model 
(Department  of  the  Air  Force,  1993c:  15) 

Analysis.  In  this  phase  of  ISD,  occupational  requirements  are  analyzed  to 
determine  training  requirements.  It  is  important  to  understand  that  job  requirements  do 
not  necessarily  translate  into  training  objectives.  Rather,  training  should  be  developed 
only  for  those  job  tasks  which  trainees  have  not  already  mastered.  Information  on  job 
tasks  is  typically  collected  through  occupational  surveys  and  observation  of  individuals 
who  are  already  performing  the  job  of  interest.  In  addition  to  occupational  analysis. 
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further  analysis  must  consider  the  needed  resources  for  conducting  training  and  the  nature 
of  the  training's  target  audience  (Department  of  the  Air  Force,  1993c,  43-60). 

Design.  This  phase  begins  with  developing  learning  objectives  which  will 
satisfy  the  trairting  requirements  determined  in  the  analysis  phase.  The  next  step  is 
development  of  tests  to  measure  student's  achievement  of  the  learning  objectives.  Review 
of  existing  instructional  material,  selection  of  instructional  methods  and  media,  and 
development  of  an  instructional  information  management  system  should  also  accomplished 
in  the  design  phase  (Department  of  the  Air  Force,  1993c,  61-110). 

Development.  After  the  instructional  system  has  been  designed,  a  course 
syllabus  is  developed.  The  syllabus,  otherwise  known  as  the  plan  of  instruction  (POI), 
identifies  course  objectives,  sequence  and  duration  of  instruction,  instructional  methods, 
instructional  materials,  equipment,  instructor  guidance,  and  lesson  plans.  Once  the  POI 
has  been  developed,  resources  such  as  lesson  plans  and  instructional  materials  must  be 
developed  and  validated.  Validation  of  instruction  is  accomplished  through  internal 
review  of  the  instructional  system,  and  by  testing  the  system  on  actual  students.  Any 
problems  revealed  during  validation  should  be  corrected  prior  to  progressing  to  the  next 
phase  of  development.  Finally,  the  instructional  management  information  system  designed 
in  the  previous  phase  is  installed  in  the  development  phase  (Department  of  the  Air  Force, 
1993c,  111-135). 

Implementation.  The  step  after  validation  of  the  instructional  system  is  to 
implement  and  maintain  the  new  system  under  real  world  conditions.  Operation  and 
maintenance  of  the  system  will  require  that  the  organizational  functions  of  management, 
support,  administration,  and  delivery  are  effective.  Furthermore,  course  evaluation  and 
customer  feedback  are  of  great  importance  in  determining  if  a  return  to  other  phases  of 
development  is  warranted  (Department  of  the  Air  Force,  1993c,  136-177). 
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Evaluation.  Evaluation  of  the  instructional  system  should  occur 


throughout  all  phases  of  development  and  implementation.  In  fact,  an  evaluation  plan  is 
devised  early  in  the  ISD  process  and  is  revised  during  each  phase  of  development. 
Continuous  evaluation  is  intended  to  assure  adherence  to  Quality  Air  Force  (QAF) 
philosophies  during  all  phases  of  instructional  systems  development  (Department  of  the 
Air  Force,  1993c,  42). 

The  Air  Force  contends  that  the  competent  application  of  ISD  will  result  in 
effective  and  cost  efficient  training  (Department  of  the  Air  Fwce,  1993c,  14).  However, 
competent  application  of  ISD  for  AMO  training  involves  a  large  number  of  organizations 
and  a  broad  base  of  job  skills  and  knowledge.  As  a  consequence,  fielding  an  effective 
training  system  is  extremely  complex  and  requires  a  systems  perspective. 

A  Systems  Perspactive.  According  to  Hayes,  the  military’s  efforts  to 
implement  Instructional  Systems  Development  (ISD)  have  been  hampered  because  "many 
training  developers,  administrators,  and  training  device  designers  continue  to  operate 
solely  in  their  own  area  without  an  awareness  of  the  influence  of  other  groups  on  their 
products"  (1992: 258),  In  the  AMO  training  system,  these  groups  of  people  consist  of 
individuals  such  as  AMMOC  students,  their  supervisors,  people  assigned  to  the 
organizations  and  major  commands  to  which  they  are  assigned,  and  the  personnel  who  are 
part  of  the  training  system. 

Acceding  to  Hayes,  such  a  wide  spectmm  of  stakeholders  results  in  a  diversity  of 
stakeholder  interests  which,  in  many  cases,  are  conflicting.  For  example,  an  AMO 
assigned  to  a  nuclear  munitions  storage  activity  at  a  B-52  wing  has  different  training  needs 
than  an  AMO  assigned  to  a  propulsion  maintenance  activity  in  C-17  wing.  Differing 
stakeholder  interests  are  not  limited  to  customers  of  training.  Hayes  points  out  that  at  the 
Department  of  Defense  level  of  management,  reducing  the  budget  is  the  primary  goal, 
while  training  providers  are  interested  in  student  throughput  Because  of  such  potentially 
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conflicting  goals,  and  a  corresponding  lack  of  a  "true  systems  perspective,"  training 
systems  often  become  fragmented  and  suboptimized  (Hayes,  1992:  261-267). 

The  Training  System 

In  a  search  for  improvement  opportunities,  an  understanding  of  the  global  training 
system  to  which  AMMOC  belongs  is  necessary.  What  follows  is  a  broad  description  of 
the  organizations  and  processes  that  define  this  system.  The  system  described  is 
composed  of  the  activities  within  the  82nd  Traiiung  Group  which  are  pertinent  to 
AMMCX)  training  operations  and  course  development,  the  organizations  with  which  these 
activities  interact,  and  AMMOCs  external  customers. 

Because  of  the  ongoing  levisicxi  of  technical  training  policy  and  the  unavailability 
of  official  documentation,  a  clear  description  of  the  training  system  was  difficult  to 
develop.  This  systems  perspective  is  based  on  published  regulations,  drafts  of  regulations 
pending  publication,  the  expertise  of  training  practitioners  assigned  to  Second  Air  Force 
(2AF)  and  the  82nd  Training  Group,  and  the  researchers'  background  in  technical  training. 
The  following  description  has  been  reviewed  by  the  82nd  Training  Group  practitioners 
who  are  cited  throughout  the  remainder  of  this  chapter. 

Structure  of  Training 

Air  Education  and  Training  CcHiimand  (AETC)  is  responsible  for  recruiting, 
training,  and  educating  Air  Force  military  and  civilian  personnel.  Second  Air  Force  (2AF) 
is  the  component  of  AETC  responsible  for  technical  training  and  consists  of  four  training 
wings  located  throughout  the  United  States  (Air  Force  Magazine,  1994:  63-64). 

AMMOC  is  assigned  to  the  82nd  Training  Wing  at  Sheppard  AFB,  Texas.  Figure  2-2 
below  depicts  the  structure  of  training  from  AETC  Headquarters  down  to  AMMOC. 
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The  Customer.  In  accordance  with  ISD,  the  justification  for  AMMOC  is  based 
on  its  customers'  needs  for  training.  Consistent  with  Thomas'  assertion,  we  define 
AMMOC's  external  customers  as:  1)  its  students;  and  2)  other  people  who  benefit  from 
the  trainees'  enhanced  job  performances  (1992;  75).  AMMOC's  customers  are  captive. 
AMMOC  is  a  mandatory  requirement  for  an  officer's  progression  through  the  aircraft 
maintenance  officer  career  field;  no  other  official  source  of  training  is  available. 

As  noted  by  Bair  and  Gatewood  (1982),  AMMOC's  customers  are  a  diverse 
population  of  individuals.  First,  officers  enter  the  AMO  career  field  with  a  variety  of 
educational  and  occupational  experiences.  Some  have  no  aircraft  maintenance  experience 
while  others  possess  substantial  hands-on  aircraft  maintenance  experience.  Similarly,  the 
educational  backgrounds  of  AMMOC  students  are  also  diverse.  While  the  Air  Force 
desires  AMOs  with  educational  backgrounds  in  either  engineering,  management,  or  the 
physical  sciences  (Department  of  the  Air  Force,  1991b:  11-15-1 1-16),  these  desires  do 
not  constitute  firm  educational  requirements.  In  fact,  occasionally  AMMOC  encounters 
students  who  possess  degrees  in  academic  disciplines  such  as  psychology  or  English. 
Because  of  the  variety  of  missions  and  weapons  systems  associated  with  the  different 
major  commands,  a  diversity  of  job  requirements  exists  among  the  population  of  AMOs. 
Therefore,  providing  exactly  what  the  customer  needs  is  an  extremely  demanding 
challenge  for  AMMOC. 

According  m  ISD,  customer  training  needs  are  defined  by  knowledge  and  skill 
requirements  for  the  job,  and  upon  the  aptitudes  of  the  trainees  (Department  of  the  Air 
Force,  1993c:  18).  Training  curriculum  is  based  upon  task  and  knowledge  requirements 
which  are  agreed  upon  by  the  Air  Force  career  field  manager  (AFCFM)  and  major 
command  functional  managers  (MFMs)  primarily  during  Utilization  and  Training 
Workshops  (U&TW)  (Department  of  the  Air  Force,  1994c,  24-25). 
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The  U&TW  is  a  forum  that 


Utilization  and  Training  Workshop  iU&TW) 
assembles  expertise  throughout  the  Air  Force  to  determine  educational  and  training 
requirements  for  the  AMO  career  field,  and  to  evaluate  current  methods  of  satisfying  these 
requirements  (Department  of  the  Air  Force,  1994a:  4).  Workshop  participants  include  the 
AMO  AFCFM,  MFMs,  AMMOC's  training  manager,  the  Air  Force  Manpower  and 
Personnel  Center  (AFMPC),  the  Air  Force  Occupational  Measurement  Squadron,  the  Air 
Force  Institute  of  Technology  (AFTT),  and  other  representatives  from  AMMOC  and  the 
82nd  Training  Group  (Department  of  the  Air  Force,  1994d).  U&TWs  are  conducted  on 
an  "as-needed"  basis  (Department  of  the  Air  Force,  1994a:  4).  The  most  recent  U&TWs 
were  conducted  in  1987  and  1994  (Haynes,  1994). 

The  primary  product  of  the  U&TW  is  the  Career  Field  Education  and  Training 
Plan  (CFETP),  a  document  that  charts  the  educational  and  training  requirement  for 
members  of  a  particular  career  field  throughout  their  entire  careers  (Department  of  the  Air 
Force,  1994a:  Atch  1,  Part  I).  A  major  component  of  the  CFETP  is  the  Course  Training 
Standard  (CTS)  which  identifies  the  major  subject  areas,  specific  training  tasks,  and 
proficiencies  that  must  be  achieved  by  AMMOC  students  (Department  of  the  Air  Force, 
1994c:  44-48).  Table  2-2  below  presents  typical  training  tasks  contained  within  the  1992 
AMMOC  CTS. 

Table  2-3.  Typical  AMMOC  Training  Objectives 
(Department  of  Air  Force,  1993b:  4-9) 

Identify  facts  about  nuclear  theory 
Identify  facts  about  powered  support  equipment 
Identify  facts  about  maintenance  officer  additional  duties 
Identify  responsibilities  of  maintenance  production  managers 
Identify  principles  of  aircraft  maintenance  productions  planning 


Essentially,  the  CTS  is  a  contract  between  AMMOC  and  the  numerous 
organizations  which  rely  on  AMMOC  for  training,  and  should  in  fact  reflect  the  customers' 
training  needs.  Identification  of  customer  requirements  may  come  from  a  number  of 
sources  such  as  occupational  survey  reports,  customer  feedback  collected  by  the  training 
group,  and  members  of  the  U&TW  such  as  Air  Fotcc  and  major  command  representatives. 
Once  established,  modification  of  the  CTS  requires  the  approval  of  the  AFCFM  and 
coordination  of  MFMs .  Following  the  U&TW,  the  training  manager  is  responsible  for 
preparing  and  obtaining  approval  of  these  documents  (Department  of  the  Air  Force, 

1994c:  20-45). 

Development  of  AMMOC  Curriculum.  In  accordance  with  the  CTS,  the  82nd 
Training  Group  is  tasked  with  designing  a  course  that  will  satisfy  the  knowledge  and  skill 
requirements.  In  the  process  of  training  development,  two  other  course  control 
documents  are  derived  from  the  CTS-the  Course  Chart  (CC)  and  the  Plan  of  Instruction 
(POI)  (Department  of  the  Air  Force,  1994c:  64-  78). 


Table  2<4.  Excerpt  from  Course  Chart 

_ (Department  of  the  Air  Force,  1993a:  4) _ 

Course  Material  -  UNCLASSIFIED 

Block  IX  -  Production  Planning  and  Problem  Solving  66  Hours  TT 

Monthly  Aircraft  Utilization  Schedule  (11  hrs);  Emergency  War  Order  Plaiming  (8  hrs); 
Weekly  and  Daily  Aircraft  Utilization  Scheduling  with  DCM  standup  Briefings  (46  hrs); 
Course  Critique  and  Graduation  (1  hr).  End-of-Course  Appointments  (2  hrs). 
(Equipment  Hazards  and  Personnel  Safety  Integrated  with  above  Subjects) 

68  Hours  Total 


The  CC  defines  the  sequence  of  subject  areas,  the  duration  of  instruction  for  each 
training  objective,  and  the  course's  overall  length.  The  duration  of  the  training  for  each 
training  objective,  and  the  course  as  a  whole,  are  defined  by  AMMOC  and  the  training 
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group,  and  should  be  based  on  the  appropriate  time  required  to  provide  the  instruction 
(Haynes,  1994).  Table  2-4  presents  a  portion  of  the  AMMOC's  CC 


The  final  course  control  document  is  a  two-part  Plan  of  Instruction  (POI).  Part  I 
of  the  POI  is  a  course  syllabus  derived  from  the  CC  and  CTS.  It  defines  for  each  major 
subject  area:  the  specific  training  objectives;  specific  lesson  content  and  duration  for  each 
objective;  and  the  training  methods,  instructional  materials  (i.e.,  visual  aids,  student 
handouts),  training  devices,  and  instructional  guidance.  Part  II  of  the  POI  is  a  teaching 
guide  which  instructors  follow  while  providing  training  (Department  of  the  Air  Force, 
1994c:  64-78).  The  table  below  presents  a  typical  excerpt  from  the  POI  Part  I. 


Table  2-5.  Excerpt  from  Plan  of  Instruction  Part  I 
(Department  of  the  Air  Force,  1993e:  9) 


5.  Maintenance  Production  Manager's  Responsibilities  4 

a.  Identify  responsibilities  of  maintenance  production  managers.  (4) 

CTS:  2Qg  Meas:  W 

(1)  Squadron  Commander  Responsibilities 

(2)  Maintenance  Supervisor  Responsibilities 

(3)  Branch  Chief  Responsibilities 

(4)  Production  Superintendent's  Responsibilities 

(5)  Flight  line  Expediter's  Responsibilities 

(6)  Shop  Chief  Responsibilities 


82nd  Training  Group.  Development  of  training  primarily  involves  four  agencies 
within  the  82nd  Training  Group:  AMMOC,  its  respective  Training  Development  Element 
(TDE),  Training  Plans,  and  Evaluation.  AMMOC,  TDE,  and  Training  Plans  are  all 
assigned  to  the  362nd  Training  Squadron.  Within  the  362nd  Training  Squadron, 

AMMOC  and  TDE  are  aligned  under  the  Maintenance  Management  Flight,  and  Training 
Plans  is  assigned  to  Operations.  Evaluation  is  a  staff  function  aligned  under  the  82nd 
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Weapons  Systems 
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Figure  2-3.  Organizational  Chart  of  Pertinent  82nd  Training  Group  Activities 
(Department  of  the  Air  Force,  1994b) 

Training  Group.  Figure  2-3  presents  an  organizational  chart  of  the  pertinent  82nd 
Training  Group  activities.  We  are  unaware  of  any  published  policy  that  specifically 
delineates  the  responsibilities  among  Training  Plans,  Training  Development,  AMMOC, 
and  Evaluation.  Nonetheless,  the  following  organizational  responsibilities  were  obtained 
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through  a  series  of  informal  telephone  and  personal  conversations  with  82nd  Training 
Group  personnel.  These  responsibilities  are  not  all  inclusive,  but  are  the  most  relevant 
concerning  AMMOC  training  development  and  operation. 

Aircraft  Maintenance  and  Munitions  Officers  Course  (AMMQCl. 

AMMOC  is  the  "point  of  contact"  in  the  training  of  AMOs.  AMMOC  delivers  the 
training  to  the  customers.  However,  AMMOC  is  responsible  for  far  more  than  product 
delivery.  Except  for  development  of  the  CTS,  AMMOC  is  responsible  for  virtually  all 
course  develq)ment  including  researching,  plamting,  developing,  evaluating,  and  updating 
course  materials  such  as  lesson  plans,  student  handouts,  tests,  and  audiovisual  materials. 
Furthermore,  AMMOC  is  responsible  for  maintaining  student  records  and  counseling 
students.  The  AMMOC  instructor  staff  consists  of  16  instructors  who  must  accomplish 
the  previously  nsentioned  responsibilities  for  basic  AMMOC  and  three  short  courses 
(Dalton,  1994). 

Training  Development  Element.  The  Training  Development  Element  consists 
of  three  pec^le  and  is  aligned  under  the  Maintenance  Management  Flight.  Like  the 
training  manager,  a  single  traiiung  specialist  is  allocated  to  the  oversight  of  the  aircraft 
maintenance  officer,  maintenance  data  systems  analysis,  and  maintenance  scheduling 
career  fields.  TDE  oversees  development  of  course  control  documents  such  as  the  Course 
Chart  (CC)  and  the  Plan  of  Instruction  Part  I.  In  coordination  with  AMMOC  personnel, 
Traiiung  Development  establishes  the  time  and  emphasis  placed  on  each  unit  of  study,  the 
continuity  between  units,  and  the  level  to  which  students  should  be  trained  on  each  item. 
Training  Development  also  coordinates  course  control  documents  through  the  Training 
Manager  to  other  agencies  to  obtain  approval.  Additionally,  Training  E>evelopment  assists 
AMMOC  with  the  preparation  of  lesson  plans,  tests,  student  handouts,  and  audiovisual 
materials.  Finally,  Training  Development  assists  AMMOC  in  resolving  training 
deficiencies  (Haynes,  1994). 
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Training  Plans  and  the  Training  Manager.  The  Chief  of  Training  Plans  is 
assigned  to  the  Director  of  Operations  in  the  362nd  Training  Squadron.  The  Chief  of 
Training  Plans  oversees  the  development  and  planning  of  training  for  more  than  200 
technical  training  courses  within  the  362nd  Training  Squadron.  The  Chief  of  Training 
Plans  supervises  nine  training  managers  who  manage  the  technical  training  development 
and  planning  for  a  multitude  of  aircraft  maintenance  career  fields  (Shoffner,  1994). 

The  training  manager's  responsibilities  regarding  AMMOC  primarily  involve 
oversight  of  course  training  development  and  scheduling  of  students  for  the  courses  under 
his  span  of  control.  As  shown  in  Figures  2-4  and  2-5,  the  training  manager  is  an  important 
link  in  communications  between  AMMOC  and  the  remainder  of  the  training  system.  The 
training  manager  interacts  with  Second  Air  Force  (2AF),  the  Air  Force  Manpower  and 
Personnel  Center  (AFMPC),  the  Air  National  Guard  (ANG),  and  the  Air  Force  Reserve 
(AFRES)  in  managing  the  flow  of  students  through  AMMOC.  He  is  also  an  intermediary 
between  AMMOC  and  the  AFCFM  and  MFMs  in  the  development  of  the  CTS  and 
CFETP.  The  training  manager  who  oversees  training  for  the  AMO  career  field  is  also 
responsible  for  managing  training  for  the  maintenance  data  systems  analysis  and  the 
maintenance  scheduling  career  fields.  In  total,  AMMOC's  training  manager  is  responsible 
for  the  oversight  of  eleven  courses  (Lawlor,  1994). 

In  developing  class  schedules,  the  training  manager  primarily  interacts  with  2AF 
through  an  on-line  computer  based  scheduling  system  called  Simulated  Modeling  for 
Allocation  of  Resources  for  Training  (SMART)  (Lawlor,  1994).  Through  SMART, 
forecasts  of  Trained  Personnel  Requirements  (TPR)  are  provided  to  the  training  manager 
by  2AF.  The  training  manager  is  tasked  to  build  course  schedules  that  can  accommodate 
the  forecast  student  flow  within  the  constraints  of  the  courses.  The  schedule  for  each 
AMMOC  class  specifies  class  start  and  graduation  dates,  and  the  number  of  students  in 
each  class. 
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2AF  coordinates  class  scheduling  with  AFMPC,  ANG,  and  AFRES  through 
another  automated  system  called  the  Air  Force  Training  Management  System  (AFTMS) 
(McLaughlin,  1994).  ANG,  AFRES,  and  AFMPC  provide  2AF  with  TPR  forecasts, 
which  2AF  subsequently  forwards  to  the  training  manager  through  SMART,  The  class 
schedule  built  by  the  training  manager  is  provided  to  2AF  through  SMART,  and  then  to 
the  other  organizations  through  AFTMS.  At  this  point,  the  ANG,  AFRES,  and  AFMPC 
assign  trainees  to  the  classes  allocated  by  the  training  manager  (Shoffher,  1994).  It  should 
be  noted  that  these  organizations  can,  and  sometimes  do,  informally  communicate  over  the 
telephone.  Presented  in  Table  2-4  below  is  a  model  of  the  communications  involved  in 
student  scheduling. 


Training  managers  also  communicate  with  Air  Force  career  tield  managers  and 
major  command  functional  managers  in  developing  and  obtaining  approval  for  the  CTS. 
Normally,  the  CTS  is  agreed  upon  by  these  managers  during  U&TWs.  Once  the 
wOTkshop  is  complete,  the  training  manager  is  responsible  for  developing  a  Course 
Training  Standard  (CTS)  and  a  Career  Field  Education  and  Training  Plan  (CFETP).  Once 
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these  documents  are  approved,  training  managers  do  not  normally  pursue  modification  of 
them  outside  of  the  U&TW  forunx  Training  managers  regard  the  Air  Force  career  field 
manager  as  AMMOC's  primary  customer  representative,  and  usually  do  not  interact  with 
the  major  command  functional  managers  (Lawlor,  1994).  Presented  in  Figure  2-5  below 
is  a  model  of  the  lines  of  communications  with  the  AFCFM  and  MFMs  in  developing  the 
CTS.  Because  of  the  infrequent  nature  of  U&TWs  (an  interval  of  seven  years  between  the 
most  recent  ones),  we  have  excluded  U&TWs  from  this  model  which  is  intended  to  depict 
communications  under  routine  circumstances.  The  dashed  lines  depict  lines  of 
communication  which  are  rarely  used. 


Figure  2>5.  Commuiucations  Model  for  CTS  Development 


Evaluation.  Training  groups  are  required  to  maintain  a  Training  Quality 
Assessment  Program  (TQAP)  (Department  of  the  Air  Force,  1994e:  1).  As  depicted 
earlier  in  Figure  2-2,  Evaluation  is  an  82nd  Training  Group  staff  function.  Evaluation 
suppons  more  than  3(X)  resident  and  exponable  courses  administered  by  the  82nd 


2-23 


Training  Group.  This  program  is  designed  to  provide  meaningful  feedback  to  technical 
training  courses  from  their  customers  through  a  number  of  internal  and  external  customer 
feedback  mechanisms  (Speights,  1994). 

Internal  feedback  is  gathered  to  help  assess  the  effectiveness  of  the  instructional 
system,  improve  support,  and  identify  exemplary  people  and  resources.  Internal  feedback 
is  typically  derived  from  student  critiques,  instructional  reviews,  student  counseling,  and 
training  statistics.  Student  attrition  and  failure  rates  are  examples  of  statistics  that  may  be 
of  value  in  identifying  potential  problems  with  training  (Department  of  the  Air  Force, 
1994e:  1).  End-of-course  critiques  consist  of  two  products.  One  is  a  42-item 
questioimaire  administered  to  each  graduating  student  to  solicit  feedback  not  only  on  the 
quality  of  instruction,  but  also  on  base  support  services.  The  other  product  consists  of 
criticisms  and  improvement  ideas  generated  by  graduating  classes  during  group 
brainstorming  sessions  (Speights,  1994).  Furthermore,  individual  students  are  free  to 
submit  course  critiques  at  any  time  (Dalton,  1994). 

External  feedback  is  gathered  to  determine  the  abilities  of  graduates  to  perform 
their  jobs,  and  to  determine  if  revision  of  training  is  needed.  The  following  are  common 
forms  of  external  feedback  gathered  for  technical  training  courses  (Department  of  the  Air 
Force,  1994c:  2). 

Occupational  Surveys  Report.  Occupational  survey  reports  (OSRs)  are 
usrful  not  only  in  designing  future  training,  but  also  in  assessing  the  usefulness  of  current 
training  (Department  of  the  Air  Force,  1994e;  2).  OSRs  can  provide  information  such  as 
what  tasks  are  required  to  perform  a  particular  job,  who  performs  them,  their  difficulty, 
and  how  often  they  are  performed.  This  information  may  be  obtained  frvm  a  variety  of 
methods  such  as  questionnaires,  observations,  interviews,  and  review  of  publications. 
Occupational  surveys  are  conducted  by  the  Occupational  Measurement  Squadron  at 
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Randolph  AFB,  Texas  (Department  of  the  Air  Force,  1993d;  46).  The  most  recent  OSRs 
potaining  to  AMMOC  were  conducted  in  1994  and  in  1980  (Wilcox,  1994). 

Training  Evaluation  Report  (TER).  TERs  provide  an  overall  assessment 
of  training  for  a  particular  course  by  considering  all  available  feedback,  both  external  and 
internal.  External  feedback  is  collected  from  graduates  and  their  supervisors  some  period 
after  graduation  in  the  form  of  Training  Quality  Reports  (TQRs),  Field  Evaluation 
QuestionnaiTes  (FEQ&),  Graduate  Assessment  Surveys  (GASs),  and  field  interviews. 
Internal  measures  included  in  the  report  typically  consist  of  graduation,  failure,  and 
elimination  rates  (Speights,  1994).  TERs  are  not  conducted  on  an  ongoing  basis  for  any 
particular  course,  but  are  ncamally  pursued  when  there  is  an  indication  that  a  problem  may 
exist  (Department  of  the  Air  Force,  1994e:  3).  The  most  recent  training  evaluation  for 
AMMOC  was  in  1991  (Haynes,  1994). 

Training  Quality  Report  (TOR).  TQRs  are  an  avenue  for  supervisors  of 
graduates  to  identify  training  deficiencies,  and  to  identify  cases  where  too  much  training 
was  provided.  These  reports  are  initiated  in  the  field,  and  are  eventually  provided  to  the 
appropriate  training  groups  for  action.  TQRs  are  infrequently  received  from  the  field  for 
any  training  course  (Speights,  1994).  No  TQRs  pertaining  to  AMMOC  have  been 
received  in  several  years  (Haynes,  1994). 

Field  Evaluation  Questionnaires.  At  the  present  time,  FEQs  do  not  solicit 
feedback  on  a  regular  basis  for  all  courses.  Instead,  they  are  employed  when  a  particular 
course  is  the  subject  of  a  TER.  Graduates  and  their  supervisors  are  mailed  FEQs  four  to 
six  months  after  graduation.  FEQs  are  usually  based  on  the  course  training  standard 
(CTS)  and  solicit  feedback  on  specific  CTS  items.  FEQs  mailed  to  supervisors  ask 
supervisors  to  rate  the  graduates  performance  of  each  CTS  item.  FEQs  mailed  to 
graduates  ask  graduates  to  rate  the  adequacy  of  training  on  specific  CTS  items.  Graduates 
and  supervisors  are  also  asked  if  training  on  specific  CTS  items  have  been  used  by  the 
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graduates.  The  FEQs  are  then  analyzed  to  determine  what  percentage  of  graduates  use 
training  on  particular  CTS  items  (Speights,  1994;  Department  of  the  Air  Force,  1994e). 

Graduate  Assessment  Survey.  Graduate  Assessment  Surveys  are  mailed 
to  1(X)  percent  of  the  supervisors  of  graduates  from  enlisted  initial  skills  training  and  their 
supervisors.  These  surveys  consist  of  four  questions  regarding  the  attitude  and  abilities  of 
the  graduates  (Department  of  the  Air  Force,  1994e:  16).  Presently,  graduate  assessment 
surveys  are  not  used  in  assessing  AMMOC  graduates  (Speights,  1994) . 

Customer  Service  Information  Line.  The  Customer  Service  Information 
Line  (CSEL)  is  a  24  hour  telephone  line  possessed  by  Evaluation  within  each  training 
group  for  receiving  customer  inquiries  and  other  feedback  from  the  field  (Department  of 
the  Air  Force,  1994e:  2).  The  phone  number  is  publicized  in  graduate  assessment  surveys 
and  in  the  Course  Training  Standard  (CTS).  However,  graduate  assessment  surveys  do 
not  presently  pertain  to  AMMOC  (Speights,  1994),  and  the  distribution  of  the  CTS  is 
restricted  (Department  of  the  Air  Force,  1993b:  1). 

Reid  Interview.  Field  interviews  are  personal  interviews,  in  the  field  or 
over  the  telephone,  with  recent  graduates,  their  inunediate  supervisors,  or  others  who  are 
familiar  with  the  performance  of  the  graduates.  These  interviews  normally  focus  on  the 
effectiveness  and  currency  of  training  (Department  of  the  Air  Force,  1994e:  2).  Field 
interviews  are  primarily  a  tool  for  establishing  a  rapport  with  customers  in  the  field;  in 
AMMOCs  case,  field  interviews  are  not  conducted  in  numbers  great  enough  to  yield 
statistical  significance  (Speights,  1994). 

Chapter  Summary 

This  chapter  provided  background  information  needed  to  understand  the 
importance  and  relevance  of  this  research.  This  chapter  summarized  relevant  research  and 
described  the  Aircraft  Maintenance  Officer  (AMO)  career  field,  AMO  training,  and  the 
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training  system  in  which  AMMOC  exists.  Chapter  in  presents  the  methodology  for 
achieving  the  research  objective  and  answering  the  investigative  questions  presented  in 
Chapter  1. 
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m.  Methodology 


Chapter  Overview 

This  chapter  describes  the  methodologies  used  to  answer  the  investigative 
questions  presented  in  Chapter  1.  We  first  reintroduce  the  basic  issues  associated  with  this 
thesis,  and  then  introduce  the  concept  of  focused  synthesis.  Focused  synthesis  was  the 
methodology  used  to  ctxnpile  and  analyze  the  background  information  presented  in 
Chapter  Q.  After  a  brief  discussion  of  focused  synthesis,  we  describe  a  two-round  Delphi 
technique  that  was  implemented  to  obtain  the  opinions  of  AMMOC  instructors  on  the 
guidance  and  support  provided  to  AMMOC.  The  specifics  on  how  the  Delphi 
questionnaires  were  developed,  structured,  administered,  and  analyzed  are  presented. 

Background 

Past  research  concerning  AMO  training  has  generally  focused  on  AMMOC  course 
curriculum  and  customer  satisfaction,  and  suggests  a  significant  level  of  dissatisfaction 
with  AMO  technical  training  over  the  past  decade  (Bair  &  Gatewood,  1982;  1 1-12; 
Department  of  the  Air  Force,  1991a:  2;  Frisbee,  1988:  2-3).  These  approaches  were 
useful  for  analyzing  training  from  the  point  of  external  customers.  However,  to  initiate 
improvements  in  trairting  processes,  we  focused  on  internal  workings  of  the  AMO  ttaiiting 
system  with  an  emphasis  on  the  QAF  concepts  of  continuous  improvement,  customer 
focus,  and  empowerment  Recognizing  that  AMMOC  instructors  are  the  internal 
custtxners  of  the  trairting  system,  we  elected  to  acquire  their  views  on  the  guidance  and 
support  afforded  to  AMMOC  to  achieve  the  following  research  objective. 

Research  Qb|ective.  Identify  opportunities  to  improve  the  support  and 
gmdance  provided  to  the  Aircraft  Maintenance  and  Munitions  Officers  Course  (AMMOC) 
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Investigative  Questions. 

1.  What  areas  related  to  the  guidance  and  support  provided  to  AMMOC 
are  the  strongest  candidates  for  improvement  inidadves? 

2.  How  can  the  idendfied  areas  be  improved? 

Methodology 

As  IS  the  case  with  just  about  any  process  improvement  inidadve,  one  must  first 
understand  the  current  processes  and  systems  imder  invesdgadon.  To  provide  this 
understanding,  we  developed  a  descripdve  accounting  of  the  aircraft  maintenance  officer 
training  system  and  relevant  processes.  Such  a  descripdve  accounting  serves  three 
purposes.  First,  it  provides  the  authors  an  undentanding  of  how  AMMOC  and  its  support 
structure  are  organized  and  managed.  Second,  it  provides  background  informadon  for  the 
readers  of  this  thesis.  And  third,  as  is  often  the  case,  the  background  invesdgadon  itself 
serves  as  a  source  of  improvement  opportunides.  The  specific  methodology  we  selected 
for  conducting  this  background  research  is  a  mediodology  called  "focused  synthesis," 
which  is  described  in  the  next  secdon  (Majchrzak,  1984:  59). 

After  establishing  the  descripdve  accounting  of  the  training  system,  we  selected  the 
AMMOC  instructors  as  one  of  our  sources  for  improvement  opportunides.  Instructors 
were  selected  because  they  are  the  internal  customers  of  the  training  system.  We  used  the 
Delphi  technique  to  obtain  the  instructors'  ideas  and  to  establish  a  level  of  agreement  on 
die  issues  and  ideas  generated.  A  descripdon  of  Delphi  and  the  radonale  for  using  it  are 
presented  later  in  this  chapter. 

The  two  parts  of  this  methodology,  the  focused  synthesis  and  Delphi,  were  not 
independent.  The  descripdve  accoundng  of  the  training  development  process  contributed 
to  the  develqiment  of  the  quesdonnaires  in  the  Delphi  technique,  and  the  responses  from 
the  quesdonnaires  contributed  to  the  descripdve  accoundng. 
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Limitations  of  the  Methodology.  During  the  course  of  this  research,  Air 
Education  and  Training  Command  was  in  the  midst  of  reviewing  and  redefining  its  training 
policy.  This  review  further  challenged  the  development  of  our  narrative  description. 

Every  attempt  was  made  to  incorporate  the  most  recent  policies  into  this  research. 
However,  it  is  possible  that  some  of  the  policies  relevant  to  this  thesis  changed  during  or 
immediately  following  the  data  collection  for  this  study.  If  problems  cited  in  this  research 
have  been  corrected  as  a  result  of  policy  changes,  then  the  intent  of  this  research  will  have 
been  met 

It  is  also  important  to  recognize  that  we  limited  the  panel  of  Delphi  respondents  to 
the  population  of  the  AMMOC  instructor  staff.  We  chose  AMMOC  instructors  as  the 
respondents  because,  as  the  providers  of  training  and  as  the  customers  of  the  guidance  and 
support  they  are  afforded,  they  have  the  best  perspective  for  suggesting  quality 
improvements  to  the  training  process.  Additionally,  limiting  the  Delphi  panel  to  AMMOC 
instructors  restricted  the  number  of  respondents  to  a  manageable  level.  We  recognize  that 
other  credible  sources  of  training  expertise  and  improvement  ideas  were  available. 

Form  of  Results.  Chapter  IV  includes  a  detailed  discussion  of  the  Delphi 
findings  and  Chapter  V  presents  recommended  improvements.  Levels  of  agreement 
among  the  instructors  for  each  of  the  improvement  opportunities  are  also  provided  as  part 
of  Appendix  G  and  H  so  that  the  readers  may  verify  for  themselves  the  results  of  our 
Delphi  procedure.  The  remainder  of  this  chapter  presents  the  detailed  methodology  used 
in  this  research. 

Focused  Synthesis 

A  descriptive  accounting  of  the  training  development  system  is  presented  in 
Chapter  II  as  part  of  the  background  information  for  this  thesis.  This  accounting  of  the 
training  development  system  was  derived  through  a  process  known  as  "focused  synthesis." 
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Focused  synthesis  is  somewhat  akin  to  traditional  literature  reviews  by  involving 
the  selective  review  of  written  materials  and  existing  research  findings  relevant  to 
the  particular  research  questions.  However,  focused  synthesis  differs  from 
traditional  literature  reviews  by  discussing  information  obtained  from  a  variety  of 
sources  beyond  published  articles.  For  example,  a  typical  synthesis  might  include 
discussions  with  expens  and  stakeholders,  congressional  hearings,  anecdotal 
stories,  personal  past  experience  of  the  researchers,  unpublished  documents,  staff 
memoranda,  and  published  materials.  (Majchrzak,  1984: 59) 

In  our  particular  "focused  synthesis,"  we  relied  on  Air  Force  regulations,  drafts  of 
pending  Air  Force  regulations,  informal  discussions  with  training  practitioners, 
unpublished  documents,  researchers’  backgrounds  in  training,  and  results  of  the  Delphi 
procedure.  Our  synthesis  of  a  descriptive  model  was  particularly  difficult  due  our 
geographical  separation  from  AMMOC,  and  the  unavailability  of  up-to-date  regulations. 
Therefore,  our  synthesis  relied  heavily  on  one  researchers'  travel  to  Sheppard  AFB,  Texas, 
where  information  was  gathered  through  discussions  with  training  practitioners.  Informal 
conversations  with  82nd  Training  Group,  Second  Air  Force,  and  Air  Force  Manpower  and 
Personnel  Center  were  also  important  bases  for  the  descriptive  model  presented  in  Chapter 
n.  As  formulation  of  the  description  progressed,  portions  of  it  were  mailed  to  82nd 
Training  Group  practitioners  for  their  review  for  accuracy.  Corrections  were  made 
accordingly. 

Delphi  Technique 

According  to  Linstone  and  Tuioff,  the  Delp.b:  technique  is  a  controlled 
commuitication  process  for  developing  consensus  among  a  panel  of  experts  (1975:  3). 
"Project  Delphi,"  an  Air  Force  sponsored  research  project  conducted  by  the  RAND 
Corporation  in  the  1950's,  marked  the  first  known  use  of  this  technique.  The  objective  of 
this  initial  application  of  Delphi  was  to  ascertain  the  consensus  opinions  of  experts  in 
predicting  Soviet  targeting  of  U.S.  industrial  sites.  This  group  of  experts  was 
administered  a  series  of  questionnaires  to  obtain  their  opinions.  Between  rounds  of 
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questionnaires,  the  experts  were  exposed  to  the  anonymous  responses  of  the  other 
participants.  This  process  established  a  reliable  consensus  of  opinion  among  the  experts. 
Today,  the  Delphi  technique  is  a  common  tool  employed  by  military  and  civilian 
researchers  for  obtaining  the  consensus  opinion  of  experts  (linstone  &  Turoff,  1975:  10). 

The  general  fonn  of  the  Delphi  technique  is  still  the  same  today.  It  involves 
acquiring  the  opinions  of  experts,  selectively  providing  the  experts  feedback  on  the 
responses  of  other  participants  while  maintaining  a  level  of  anonymity,  and  then  polling  the 
experts  again  for  their  opinimis  (Linstone  &  Turoff,  1975: 5).  Klassen  and  Whybark 
highlight  the  importance  of  the  feedback  process  by  noting  that,  "the  feedback  to  th^ 
respondents  at  each  iteration  encourages  them  to  evaluate  and  weigh  the  opinions  of 
others  in  forming  their  next  response"  (1994:  386-387). 

The  major  difference  between  the  original  Delphi  technique  and  the  techniques  of 
today  is  in  the  expanding  application  of  the  Delphi  technique.  If  the  Delphi  technique  is 
viewed  as  a  communication  process,  as  proposed  by  Linstone  and  Turoff,  then  "there  are 
few  areas  of  human  endeavor  which  are  not  candidates  for  application  of  Delphi"  (1975: 

4).  The  following  prior  uses  of  Delphi  demonstrate  its  broad  application: 

•  Planning...of  curriculum  development 

•  Delineating  the  pros  and  cons  associated  with  potential  policy  options 

•  Developing  causal  relationships  in  complex...social  phenomena 

•  Exposing  priorities  of  personal  values,  social  goals  (Linstone  &  Turoff, 

1975:  4) 

Linstone  and  Turoff  go  on  to  explain  that  the  application  itself  should  not  be  the 
determining  factor  for  using  Delphi.  It  is  often  the  circumstances  surrounding  the  problem 
(x:  issue  which  justify  the  use  of  Delphi,  such  as: 
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[1]  The  problem  does  not  lend  itself  to  precise  analytical  techniques  but  can 
benetit  from  subjective  judgments  on  a  collective  basis 

[2]  The  individuals  needed  to  contribute  to  the  examination  of  a  broad  or  complex 
problem  have  no  history  of  adequate  communication  and  may  represent  diverse 
backgrounds  with  respect  to  experience  or  expertise 

[3]  More  individuals  are  needed  than  can  effectively  interact  in  a  face-to-face 
exchange 

[4]  Hme  and  cost  make  frequent  group  meetings  infeasible 

[5]  The  efficiency  of  face-to-face  meetings  can  be  increased  by  a  supplemental 
group  commuiucation  process 

[6]  Disagreements  among  individuals  are  so  severe  or  politically  unpalatable  that 
the  communication  process  must  be  refoeed  and/or  anonymity  assured 

[7]  The  heterogeneity  of  the  participants  must  be  preserved  to  assure  validity  of 
the  results,  i.e.,  avoidance  of  domination  by  quantity  or  by  strength  of  personality 
("bandwagon  effect")  (Linstone  &  Turoff,  1975: 4) 

Though  each  of  the  properties  above  may  have  partially  justified  using  the  Delphi 
technique  for  this  research,  the  primary  properties  which  justiffed  its  use  are  1  and  4 
above. 


Armstrong  identifies  several  advantages  to  using  Delphi.  First,  Delphi  lends  itself 
to  administration  though  the  mail-a  significant  advantage  for  us  because  we  are 
geographically  separated  from  the  instructors.  Second,  Delphi  is,  "a  technique  that  is 
acceptable  to  organizations.  It  sounds  fancy,  yet  the  users  can  understand  it"  (Armstrong, 
1985: 1 19).  Finally,  Armstrong  cites  other  studies  that  provide  evidence  that,  "Delphi  is 
more  accurate  than  traditional  group  meetings"  (1985:  1 19). 

Implementation  of  the  Delphi  Technique 

To  complete  our  research  objective  and  answer  the  associated  investigative 
questions,  we  developed  and  administered  a  two-round  Delphi  procedure.  Our  Delphi 
procedure  was  adapted  from  an  Air  Force  Institute  of  Technology  (AFIT)  thesis  by  Kettell 
and  Ziegler  (1992).  The  purpose  of  our  application  of  the  Delphi  technique  was  to 
identify,  from  the  AMMOC  instructors'  perspective,  what  areas  of  the  training  system 
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external  to  AMMOC  could  be  improved,  and  to  solicit  ideas  on  how  n>  implement 
improvements. 

Participants.  The  Delphi  was  administered  to  18  AMMOC  instructors  in  the 
first  round,  and  17  AMMOC  instructors  in  the  second  round.  One  of  the  instructors  left 
AMMOC  between  the  two  rounds.  The  AMMOC  instructors  had  varying  degrees  of 
maintenance  and  instructional  experience.  Maintenance  experience  ranged  fi’om  two 
instructors  with  no  practical  maintenance  experience  to  seven  instructors  with  six  or  more 
years  of  maintenance  experience.  Cumulative  instructional  experience  ranged  from  six 
instructors  with  less  than  two  years  experience  to  six  instructors  with  more  than  6  years 
experience.  Relative  to  AMMOC  instractional  experience,  nine  instructors  had  less  than 
two  years  experience  and  three  had  more  that  6  years  experience.  Of  the  seventeen 
instructors  who  participated  in  the  second  round,  six  were  civilian  and  1 1  were  officers.  It 
is  also  worth  noting  that  nine  of  the  seventeen  participants  had  filled  non-instructional 
maintenance  positions  within  the  last  four  years 

Sponsorship.  To  generate  a  high  level  of  interest  in  our  research,  we  pursued 
and  obt^ed  sponsOTship  from  the  Maintenance  Policy  Division  at  Headquarters,  United 
States  Air  Fwce  in  Washington,  DC.  On  our  behalf.  Maintenance  Policy  Division  notified 
the  82nd  Group  Commander  of  this  research  and  requested  his  cooperation  in  allowing  the 
administration  of  questionnaires  and  the  conduct  of  informal  interviews  with  his  group 
staff.  Maintenance  Policy  Division's  letter  to  the  82nd  Group  Commander  is  presented  in 
Appendix  B. 

Advance  Letter  of  Notification.  Emory  and  Cooper  suggest  that  an  advance 
notification  may  improve  response  rates  (1991;  334).  We  provided  such  notification  a 
week  in  advance  of  mailing  out  the  first  round  questionnaires  to  the  uistructors.  The 
letter,  presented  in  Appendix  C,  informed  the  instructors  of  their  selection  for  this 
procedure  and  introduced  them  to  the  subject  of  our  research.  We  also  attempted  to 
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goierate  a  genuine  interest  in  the  research  by  emphasizing  that  the  results  would  be 
presented  to  the  Air  Force.  Finally,  we  stressed  the  imponance  of  their  participation  in  the 
study. 

Development  of  the  Rrst  Round 

Our  first  priority  was  to  establish  a  basic  set  of  topics  that  would  generate  an 
interest  in  our  research  and  serve  as  a  stepping  stone  for  the  identification  of  improvement 
opportunities.  This  set  of  topics  was  derived  from  notes  taken  during  a  1992  Quality 
Improvement  Workshop  in  which  the  instruct(»rs  identified  problems  affecting  AMMOC. 
The  general  problem  areas  included  quality  measurement,  course  content,  course 
operations,  student  administration,  reference  materials,  morale,  funding,  decentralized 
ctHitrol,  equipment,  and  facilities  (AMMOC,  1992).  Within  each  problem  area  cited  were 
specific  issues  pertaining  to  the  problem  areas.  We  classified  each  of  the  specific  issues  as 
either  internal  or  external  to  AMMOC's  control.  Because  the  focus  of  this  study  is  on 
external  factors  to  AMMOC,  we  based  the  first  round  on  those  issues  that  we  determined 
through  subsequent  research  as  primarily  external  to  AMMOC's  control. 

As  suggested  by  Dillman,  we  used  a  series  of  easy  close-ended,  short  response 
questions  as  "icebreakers"  (1978: 123).  Our  questions  consisted  of  demographic 
questions  to  ascertain  the  instructional  and  maintenance  experience  of  the  instructors  (see 
Appendix  E).  We  believed  such  demographics  of  the  instructor  population  might  have 
been  useful  in  the  analysis  of  the  responses  if  significant  disparities  among  the  respondents 
existed.  The  last  three  close-ended  questions  were  designed  to  place  the  instructors  in  a 
quality  improvement  orientation  by  increasing  their  awareness  of  Quality  Air  Force 
initiatives. 

The  remainder  of  the  first  round  questionnaire  consisted  of  open-ended  questions 
intended  to  "stimulate  free  thought,  solicit  suggestions,  probe  people's  memories,  and 
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clarify  positions"  (Dillman,  1978:  87).  We  attempted  to  maintain  the  problem-seeking 
questions  as  neutral  as  possible  by  soliciting  both  strengths  and  weaknesses  of  the 
particular  areas  of  interest.  We  thought  neutral  questionnaire  items  were  needed  to  help 
assure  that  respondents  would  not  feel  threatened  or  offended  by  the  questionnaire. 

Like  Kettell  and  Ziegler  (1992),  we  presented  the  questionnaire  in  a  format  that 
shortened  its  apparent  length.  The  questionnaire  was  divided  into  four  sections,  A 
through  D,  and  then  subdivided  into  numbered  subject  areas.  Within  each  major  subject 
area,  lower  case  alphanumerics  were  used  to  further  delineate  questionnaire  items  where 
required  (see  Table  3-1).  The  apparent  length  of  the  questionnaire  was  further  reduced  by 
printing  it  on  both  sides  of  the  paper. 

Table  3-1.  Numbering  Format  for  Round  One. 

Section  B:  General  Quality  Issues 
1.  CUSTOMER  ORIENTATION. 

a.  Who  is  AMMOC's  primary  customer? 

b.  How  does  AMMOC  communicate  with  its  primary  customer? 


Questionnaire  Pretesting.  As  suggested  by  Emory  and  Cooper,  the 
questionnaire  was  pretested  to  evaluate  and  refine  it  (1991;  376).  Three  copies  of  the 
original  questionnaire  were  sent  to  former  AMMOC  instructors.  They  were  asked  to 
complete  the  questionnaires  and  to  comment  on  the  structure  and  clarity  of  the  questions. 
Two  of  the  three  were  completed  and  returned.  Based  on  these  two  responses,  we 
deleted  two  topics,  facilities  and  international  officers,  and  condensed  the  areas  concerned 
with  student  scheduling  and  administrative  tasks.  Our  objective  in  implementing  these 
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changes  was  to  shorten  the  questionnaire.  Several  questionnaire  items  were  also  modified 
to  improve  their  clarity. 

First  Round  Questionnaire  Administration.  Questionnaire  administration  was 
significantly  facilitated  by  the  82nd  Training  Group  Commander's  ^pointment  of  an 
AMMCK)  instructor  as  a  point  of  contact  (POQ  for  our  research.  All  questionnaires  were 
mailed  to  the  POC  who  distributed  the  questionnaires  to  the  eighteen  participants. 
Respondents  were  asked  to  complete  and  return  the  questionnaires  within  two  weeks  of 
receipt  Each  questionnaire  was  accompanied  by  an  addressed  return  envelope  that  could 
be  used  to  return  the  responses  via  the  Air  Face  distribution  system  or  through  our  POC. 
The  POC  also  followed  up  with  the  respondents.  By  reminding  the  instructors  to  return 
the  questionnaires  and  re-emphasizing  the  importance  of  this  research,  we  believe  the 
POCs  efforts  significantly  improved  our  respoise  rate. 

Each  questionnaire  included  a  cover  letter  (Appendix  D)  from  our  sponsor,  the 
Maintenance  Policy  Division  at  HQ  USAF.  The  cover  letter,  like  the  prequestionnaire 
letter,  described  our  research  and  emphasized  that  the  instructor's  opiiuons  were  a  vital 
contribution  to  our  research.  Furthermore,  the  letter  explained,  that  as  Air  Staff- 
sponsored  research,  the  questionnaires  were  an  excellent  opportunity  for  AMMOC  to 
influence  future  technical  training  policy.  The  questionnaire  itself  (Appendix  E)  assured 
the  respondents  that  all  responses  would  remain  anonymous,  and  indicated  that  their 
participation  was  purely  voluntary. 

Development  of  the  Second  Round  Questionnaire 

Our  objective  in  developing  the  second  round  questionnaire  was  to  measure  each 
respondents  agreement  or  disagreement  with  the  ideas  expressed  by  the  other  respondents 
in  the  first  round.  This  was  a  particularly  difficult  task  because  of  the  relatively  large 
number  of  Delphi  respondents  and  the  number  of  topics  addressed  on  the  first 
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questionnaire.  To  facilitate  this  process,  the  written  conunents  from  each  question  in  the 
first  round  were  c(»npiled  into  a  single  document.  Several  issues  became  apparent 
through  the  compilation  and  analysis  of  the  first  round  responses. 

Hrst,  several  of  the  questions  were  satisfactorily  answered  which  precluded  asking 
them  again  in  the  second  round.  For  example,  in  response  to  the  question,  "In  your 
opiiuon,  how  effective  are  the  lines  of  commuiucation  between  AMMCXT  and  the  various 
arganizatitms  (Operational  MAJCOMs,  MFC,  AETC,  Air  Staff,  etc.)  involved  in  the 
system  of  training  and  training  development?,"  15  of  the  16  participants  expressed  the 
position  that  communications  need  improvement  Additionally,  12  of  these  16 
respondents  expressed  that  communications  are  extremely  poor.  Because  of  this 
overwhelming  response,  we  saw  no  need  to  re-address  this  issue  in  the  second  round. 

Second,  many  of  the  resptxises  expressed  essentially  the  same  ideas.  In  an  effort 
to  simplify  the  second  round,  these  respcmses  were  combined  into  one  idea  or  statement 
Many  of  the  responses  also  contained  multiple  ideas.  These  statements  were  separated 
and  presented  individually. 

Third,  responses  to  some  questions  overlapped.  We  took  the  liberty  to  move 
responses  to  the  areas  to  which  the  responses  were  most  applicable.  For  example,  under 
the  area  of  empowerment  we  asked  the  question,  "As  an  instructor,  what  authority  should 
you  have  which  you  currently  lack?"  Several  of  the  responses  concerned  course  control 
docun^nts,  a  topic  explicitly  addressed  in  another  section  of  the  questionnaire.  Likewise, 
other  responses  to  the  empowerment  question  were  also  addressed  in  other  sections  of  the 
questitxinaire.  To  shorten  the  length  of  the  second  round  questionnaire,  empowerment 
related  items  were  presented  in  other  sections  of  the  questionnaire  and  the  empowerment 
section  was  deleted.  Responses  that  did  not  relate  to  a  specific  area  were  moved  to  the 
general  issues  area  at  the  end  of  the  questionnaire. 
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Fourth,  as  expected  the  first  round  generated  issues  that  were  not  specifically 
addressed  in  the  first  round.  For  example,  we  did  not  specifically  pursue  barriers  to 
effective  training  in  the  first  round.  After  receiving  the  first  round  responses,  it  became 
apparent  that  many  of  the  statements  presented  could  be  classified  as  barriers  to  effective 
training.  As  a  result,  we  presented  a  list  of  these  barriers  in  the  second  round  for  the 
respondents  to  evaluate. 

One  other  modification  in  the  second  round  was  the  addition  of  a  question 
concerning  custtxner  feedback.  Between  the  tinte  we  administered  the  first  questionnaire 
and  the  second,  we  discovered  standard  feedback  mechanisms  for  obtaining  feedback  on 
technical  training  performance.  We  created  a  question  that  would  allow  each  instmctor  to 
present  their  opinion  on  how  effective  or  ineffective  the  uses  of  these  measures  and 
reptxts  were  for  providing  AMMOC  with  meaningful  feedback. 

Second  Round  Structure.  The  format  of  the  second  round  questionnaire  is 
similar  to  the  format  used  by  Kettell  and  2^egler  (1992).  The  questionnaire  topics  and 
associated  ideas  from  the  first  round  were  placed  in  a  tabular  frxmat.  For  each  idea,  the 
respondents  were  provided  with  a  choice  to  either  "strongly  agree,"  agree,"  "disagree,"  or 
"strongly  disagree"  (see  Table  3-2).  As  highlighted  by  Kettell  and  Ziegler,  this 
measurement  scale  "differs  from  the  well-known  Likert  Scale  which  uses  the  fifth  response 
'undecided'"  (1992: 4-20).  Like  Kettell  and  Ziegler,  we  wanted  the  respondents  to  provide 
a  "clear-cut"  iesponse~in  essence  to  vote  for  or  against  the  particular  ideas  (1992: 4-20). 
If  resptmdents  were  adamant  about  maintaining  a  neutral  position,  they  had  the  option  of 
not  responding,  even  though  that  option  was  not  expressly  provided. 
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Table  3-2.  Format  for  Second  Round  Feedback  Questions 


2.  CUSTOMER  ORIENTATION:  The  first  round 
responses  identified  the  individuals  or  organizations  listed 
below  as  AMMOC’s  customers.  Please  rate  your 
agreement/disagreement  with  each  as  a  customer. 
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The  "No  Response"  column  in  Table  3-3  and  found  in  the  questionnaire  in 
Appendix  G  was  added  to  the  questionnaire  after  the  original  questionnaire,  without  the 
"No  Response"  option,  was  administered  to  the  instructors.  The  reason  the  column  was 
added  after  administering  the  questionnaire  was  to  facilitate  the  tabulation  and 
presentatitxi  of  the  results. 

_ Table  3-3.  Format  for  Second  Round  Response  Tabulation _ 


2.  CUSTOMER  ORIENTATION:  The  first  round 
responses  identified  the  individuals  or  organizations  listed 
below  as  AMMOC's  customers.  Please  rate  your 
agreement/disagreement  with  each  as  a  customer. 
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Respondents  were  encouraged  to  comment  on  their  responses  and  to  provide  any 
additional  thoughts.  To  accommodate  this  request,  space  was  provided  on  the 
questionnaire  with  instractions  to  reference  specific  questionnaire  items.  The  format  of 
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the  seccHid  round  survey  was  simplified  by  including  only  two  major  sections:  Section  A 
for  demographic  questions,  and  Section  B  for  the  remainder  of  the  questions. 

We  asked  demognq)hic  questions  again  in  the  second  round  for  the  same  reason 
they  were  iiK;luded  in  the  first  round.  These  questions  served  as  "icebreakers"  and 
provided  useful  analysis  information  in  cases  where  significant  disparities  might  have 
existed  among  the  instructs  population. 

Validity.  Validation  of  the  second  round  instrument  was  accomplished  in  two 
forms.  Brst  was  an  internal  review  by  the  researchers  for  clarity  of  each  questionnaire 
item.  Unlike  the  first  round  instrument,  the  second  round  instrument  was  not  administered 
to  former  AMMOC  instmctors  prior  to  its  administration  to  the  AMMOC  instmctor  staff 
due  to  constraints  on  the  availability  of  the  former  instructors.  Instead,  this  type  of 
validation  was  conducted  by  the  actual  respondents  concurrently  with  the  questionnaire's 
administration.  During  and  after  administratitxi  of  the  second  round  questionnaire,  the 
researchers  corresponded  with  a  few  of  the  respondents  concerning  the  usefulness  of  the 
questionnaire  items.  While  this  method  of  validation  did  not  allow  for  correction  of  the 
items  prior  to  the  questionnaire's  administration,  it  did  allow  for  the  identification  of 
unreliable  items  which  were  then  excluded  from  researchers'  consideration  in  drawing 
conclusions.  This  prcx;edure  was  worthwhile.  Respondents  did  identify  that  several  of  the 
questionnaire  items  lead  respondents  to  particular  responses.  As  a  result,  the  researchers 
reviewed  the  questionnaire  and  responses  to  identify  items  that  were  eitiier  leading,  or 
were  confusing  to  the  respondents.  Such  items  are  identified  in  Chapter  IV. 

Second  Round  Questionnaire  Administration.  Again  AMMOC's  point  of 
contact  (POQ  significantly  eased  the  administration  of  the  second  round  questionnaire. 
Only  17  questionnaires  were  mailed  for  the  second  round  because  one  of  the  instructors 
was  reassigned  from  AMMOC  without  a  replacement.  We  asked  the  instructors  to 
complete  and  return  the  questionnaires  within  10  days.  The  questionnaire  consisted  of 
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close-ended  questions  and  required  less  time  to  complete  than  the  first  round.  Our  point 
of  contact  conducted  a  follow  up  with  the  respondents  during  this  period  to  again  increase 
the  response  rate.  As  in  the  first  round,  an  addressed  envelope  accompanied  each 
questionnaire  in  case  the  instructors  wanted  to  return  their  responses  through  the  Air 
Force  distribution  system. 

The  cover  letter  for  the  second  questionnaire  (Appendix  F)  was  signed  by  the 
researchers  instead  of  the  Maintenance  Policy  Division  because  an  excellent  rapport  had 
been  established  with  the  instructors  during  the  first  round.  Our  letter  thanked  the 
instructors  for  their  participation  in  the  first  round  and  asked  them  to  participate  in  the 
second  round.  We  also  explained  that  the  second  round  consisted  of  edited  responses 
firom  the  first  round  to  shorten  the  length  of  the  questionnaire.  The  editing  process  was 
explained  to  ensure  none  of  the  respondents  would  be  offended  because  verbatim 
responses  were  not  presented.  Finally,  we  re-emphasized  the  importance  of  their 
participation  in  this  research. 

Analysis  of  the  Delphi  Results 

First  Round  Analysis.  The  analysis  of  the  first  round  was  partially  described 
earlier  as  part  of  the  description  of  the  development  process  for  the  second  round 
questicmnaire.  To  start,  we  compiled  the  written  comments  fiom  each  respondent  for  each 
question.  We  then  analyzed  the  responses  to  see  if  there  were  general  trends  within  each 
question,  dkmimon  ideas  or  thoughts  were  grouped  for  presentation  back  to  the 
respondents.  The  main  purpose  of  the  first  round  was  to  refine  the  list  of  areas  susceptible 
to  improvement  and  to  acquire  ideas  fen-  initiating  improvement.  However,  if  the 
responses  essentially  agreed  on  the  answer,  the  questions  were  not  re-addressed  in  the 
second  round. 
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Second  Round  Analysis.  The  close-ended  question  fonnat  significantly 
simplified  the  second  round  analysis.  The  responses  of  each  of  the  instmctors  were  tallied 
on  the  questionnaire  in  Appendix  G  that  had  been  modified  to  include  a  "No  Response" 
column.  Additionally,  any  comments  that  justified  dissenting  positions  or  qualified  a 
response  were  provided.  The  initial  analysis  of  these  results  consisted  of  evaluating  the 
aggregate  resptmse  for  each  question  by  comparing  the  number  of  "strongly  agree"  and 
"agree"  responses  to  the  total  number  of  participants.  After  determining  the  level  of 
agreement  on  the  responses,  then  the  level  of  emotion  was  assessed  by  examining  the  ratio 
of  "strongly  agree"  and  "strongly  disagree"  responses  to  the  total  number  of  participants. 

A  statistical  analysis  of  the  data  is  presented  in  Appendix  H.  The  four  statistical 
parameters  calculated  for  each  question  were  the  population  mean  (p),  the  population 
standard  deviation  (a),  the  percent  of  instructors  who  agreed  i%Agrtt)  statement 

presented,  and  the  percent  of  instructors  who  provided  "strong"  positions  (%strong)' 
equations  used  were. 


n 

1-. 


a  = 


A 

(  » 

2 

n^xt  + 

1  i-i 

J 

11 

1 

•'Stromf 


_  ^SmiAgnt  ^StitgDuatnt  ^  ^qq 

17 


(1) 

(2) 

(3) 

(4) 


3-16 


where  n  =  number  of  responses,  and  x  =  value  assigned  to  each  response  (strongly 
agree=4,  agree=3,  disagree=2,  strongly  disagreesl}. 

The  researchers  looked  for  general  trends  in  the  data  and  general  themes  across 
responses.  It  was  not  the  intent  of  this  thesis  to  draw  a  line  at  a  point  where  ideas  above 
the  line  should  be  implemented  and  the  (xies  below  thrown  out  The  subjective  value  and 
feasibility  of  each  idea  is  left  to  the  members  of  the  training  system  under  analysis  and  the 
readers  of  this  thesis.  The  researchers*  analysis  of  the  results  is  presented  in  Chapter  IV. 
The  subsequent  recommendations  are  presented  in  Chapter  V. 

Summary 

This  chapter  presented  the  methodology  for  answering  the  investigative  questions 
from  Chapter  I.  We  introduced  the  reader  to  focused  synthesis  as  a  method  for  describing 
die  AMMOC  training  and  training  development  process  presented  in  Chapter  n.  This 
chapter  also  explained  the  two  round  Delphi  technique  employed  to  obtain  the  opinions  of 
AMMOC  instractors  on  improvement  opportunities  and  ideas.  The  tirst  round  of  the 
Delphi  further  developed  improvement  opportunities  derived  from  a  1992  AMMOC 
(Quality  Improvement  Workshop.  The  second  round  developed  the  levels  of  agreement 
among  the  instructors  concerning  the  potential  barriers  b)  effective  training  and  other  ideas 
expressed  in  the  first  round.  Finally,  the  questionnaire  analysis  process  was  explained. 

Chapters  IV  and  V  present  the  findings  of  the  Delphi  process,  provide  an  analysis  of 
those  findings,  and  provide  recommendations  for  improving  the  policies  and  support 
provided  to  AMMOC. 
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IV.  FincHnos  and  Analysis 


Chapter  Overview 

Chiq)ter  IQ  detailed  the  Delphi  technique  applied  to  determine  what  opportunities 
exist  for  improving  the  guidance  and  support  provided  to  AMMOC,  and  how  to  take 
advantage  of  such  opportunities.  This  chapter  presents  the  findings  and  analysis  of  the 
Delphi  questionnaires  that  were  administered.  First,  response  rates  and  demographic 
profiles  of  the  participants  are  presented.  The  chapter  then  presents  the  specific  findings 
and  analysis  for  each  major  topical  area  addressed  in  the  questionnaires. 

Background 

The  purpose  of  the  first  round,  and  its  open-ended  questionnaire  format,  was  to 
obtain,  fintxn  the  AMMOC  instructors,  a  variety  of  insights  on  the  guidance  and  support 
provided  to  AMMOC,  The  results  from  the  first  round  were  compiled  and  analyzed  to 
identify  areas  susceptible  to  improvement  initiatives  and  to  identify  specific  improvement 
ideas.  During  this  first-round  analysis,  common  ideas  and  thoughts  were  consolidated  for 
presentation  to  die  participants  in  the  second  round.  In  some  cases,  positions  presented  by 
die  participants  in  the  first  round  provided  sufficient  evidence  to  conclude  certain  areas 
were  in  fact  strong  candidates  for  improvement.  If  exclusions  concerning  these 
particular  areas  could  be  drawn  from  the  first  round,  they  were  not  re-addressed  in  the 
second  round. 

The  purpose  of  the  second  round  with  its  close-ended  format  was  to  obtain  the 
AMMOC  instructors'  overall  opinion  on  guidance  and  support  issues  raised  in  the  first 
round  and  to  identify  dieir  level  of  agreement  on  the  improvement  ideas  submitted.  The 
second-round  analysis  was  based  on  the  distributix  of  the  responses  for  each  question 
and  X  die  commx  issues  supported  throughout  the  questionnaire. 


4-1 


The  raw  responses  from  the  first  round  are  not  included  in  this  thesis  to  protect  the 
identity  of  the  respondents.  Unlike  the  first  round  responses,  second  round  responses  are 
included  because  they  are  not  attributable  to  a  particular  individual.  The  second  round 
responses  are  presented  in  Appendix  G,  and  the  statistical  analysis  of  the  second  round  is 
presented  in  Appendix  H. 

Beftxe  discussing  the  findings  and  analysis,  recall  from  earlier  chapters  the 
following  investigative  questions: 

1.  What  areas  related  to  the  guidance  and  support  provided  to  AMMOC 
are  the  strongest  candidates  for  improvement  initiatives? 

2.  How  can  the  identified  areas  be  improved? 

General  Questionnaire  Results 

Response  Rates.  Sixteen  of  the  18  first-round  questionnaires  were  returned  for 
an  88.9  percent  response  rate.  More  significantly,  17  of  17  second-round  questionnaires 
were  returned  for  a  100  percent  response  rate.  Only  17  questionnaires  were  mailed  for  the 
second  round  because  one  instructor  was  reassigned  from  AMMOC  without  a 
replacement.  The  second  round  response  rate  is  especially  significant  because  it  was  in  the 
second  round  where  respondents  actually  rated  their  agreement  or  disagreement  with  the 
statements  and  ideas  presented  in  the  first  round.  The  second  round  results  are  presented 
in  Appendix  G.  It  should  be  noted  that  one  of  the  participants  in  both  rounds  of  the 
Delphi  was  not  an  active  AMMOC  instructor  at  the  time  of  the  Delphi  administration. 

This  inthvidual  was  a  former  AMMOC  instructor  who  had  been  reassigned  to  a  computer- 
based-instruction  (CBI)  activity  in  support  of  AMMOC.  Because  this  instructor 
possessed  two  years  experience  as  an  AMMOC  instructor,  and  continues  to  support 
AMMOC  in  his  new  job,  we  consider  him  a  legitimate  participant  in  this  study. 
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General  Observations.  The  responses  and  response  rates  for  the  first  and 
second  round  support  the  researchers'  belief  that  the  AMMOC  instructors  have  a  genuine 
interest  in  making  AMMOC  as  effective  and  efficient  as  possible,  and  that  they  are  an 
extremely  valuable  source  of  improvement  ideas.  It  was  also  apparent  from  the  first  round 
responses  that  many  of  the  participants  put  forth  a  substantial  level  of  effort  and  thought 
into  their  responses,  and  easily  surpassed  our  two-hour  estimate  for  completing  the  first 
round. 

Demographics.  One  of  the  benefits  of  a  100  percent  response  rate  in  the  second 
round  was  a  demographic  profile  of  the  entire  AMMOC  instructor  population.  The  data, 
is  presented  in  Appendix  G.  There  appears  to  be  a  good  cross  section  of  instructional  and 
technical  experience,  and  over  half  of  the  respondents  have  been  in  a  non-instructional 
maintenance  position  within  the  past  four  years.  There  were,  however,  two  instructors 
who  possessed  no  experience  in  aircraft  maintenance  except  in  their  roles  as  instructors. 

QAF  Orientation.  Two  questions  were  asked  during  the  first  round  concerning 
the  respondents  awareness  of  Quality  Air  Force  philosophies  and  improved  business 
practices.  The  purpose  of  these  two  questions  was  threefold.  First,  as  was  the  case  in  a 
previous  AFTT  thesis  by  Kettell  and  Ziegler,  the  questions  were  "icebreakers."  Second, 
the  questions  were  intended  to  establish  a  frame  of  reference  for  the  remainder  of  the 
questionnaire.  And  third,  they  were  used  to  determine  the  level  of  awareness  of  the 
instructors  on  QAF  principles  and  business  improvement  practices  (Kettell  &  Ziegler, 
1992: 5-3).  The  questions  and  results  concerrung  QAF  and  business  improvement 
practices  are  presented  in  Table  4-1. 

Along  the  same  lines  as  the  two  questions  in  Table  4-1,  a  third  question  was  asked 
concerning  the  respondents'  participation  in  the  1992  AMMOC  Quality  Improvement 
Workshop.  The  question  served  two  purposes.  First,  because  the  source  of  our  general 
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Table  4-1.  QAF  and  Improved  Business  Practice  Questions  and  Results 


9.  The  Quality  Air  Force  philosophy  advocates  six  basic  principles:  Leadership 
Involvement,  Dedication  to  Mission,  Respect  of  the  Individual,  Decentralized 
Organization,  Empowerment  at  the  Point  of  Contact,  and  Management  by  Fact. 
Were  you  aware  of  these  principles? 

10  Yes  6  No 

10.  In  an  efftxt  to  become  more  efficient  and  effective,  DOD  is  encouraging, 
and  in  some  cases  directing,  improved  business  practices.  Were  you  aware  of 
these  initiatives?  (Kettle  &  Ziegler,  1992:  5-3) 

12  Yes  4  No 


areas  of  interest  originated  from  the  notes  taken  during  the  workshop,  we  thought  it  might 
be  useful  to  know  if  the  population  had  changed  significantly  since  then.  We  anticipated 
that  the  population  had  changed  significantly  because  the  course  had  moved  from  Chanute 
to  Sheppard  AFB  since  the  time  the  workshop  was  held.  Second,  we  wanted  those  who 
did  participate  in  the  workshop  to  view  this  research  as  a  continuation  of  their  efforts  in 
1992  and  not  as  a  duplication  of  that  process.  The  question  and  the  results  are  presented 
in  Table  4-2. 


Table  4-2.  AMMOC  Quality  Improvement  Workshop  Participation  Results 


1 1.  In  1992,  a  quality  improvement  workshop  was  conducted  at  Chanute  AFB 
during  which  AMMOC  personnel  formulated  mission  and  vision  statements. 
Also  identified  were  potential  areas  for  improvement  that  were  beyond  the 
control  of  AMMOC.  Were  you  a  participant  in  this  workshop? 

Yes  11  No 


Customer  Orientation 

Two  questions  were  asked  in  the  first  round  pertaining  to  customer  orientation. 

a.  Who  is  AMMOC's  primary  customer? 

b.  How  does  AMMOC  communicate  with  its  primary  customer? 

The  purpose  for  asking  these  two  questions  was  to  see  what  level  of  agreement 

exists  among  AMMOC  instructors  on  who  their  primary  customers  are.  TQM  and  QAF 
suggest  service  organizations  must  first  know  who  their  customers  are  in  order  to  properly 
satisfy  the  customers'  needs. 

A  compilation  of  all  the  first  round  responses  identified  five  potential  customers: 

1.  AMMOC  students. 

2.  Supervisors  of  graduates. 

3.  Commanders  of  graduates. 

4.  MAJCOMs  and  MAJCOM  equivalents. 

5.  Foreign  governments  of  international  students. 

Recognizing  that  AMMOC  probably  does  have  more  than  one  customer,  a 

question  was  created  for  the  second  round  in  which  the  instructors  expressed  their 
agreement  or  disagreement  as  to  whether  each  of  the  above  is  a  customer  of  AMMOC 
(Appendix  G,  Question  B.2).  AMMOC  instructors  unanimously  viewed  the  supervisors, 
commanders,  major  commands  and  foreign  governments  as  customers.  However,  four 
respondents  did  not  view  the  students  as  customers. 

The  second  part  of  the  first-round  question  was  intended  to  addressed  the  means 
by  which  AMMOC  communicates  with  its  customers.  This  question  was  not  as  effective 
as  we  would  have  liked  for  two  reasons.  First,  responses  were  somewhat  dependent  upon 
whom  the  participants  identified  as  the  primary  customer  in  the  first  pan  of  the  question. 


4-5 


Secondly,  some  responses  to  the  question  addressed  communications  effectiveness,  while 
others  addressed  the  means  by  which  AMMOC  communicates. 

Despite  the  fact  that  the  question  was  interpreted  in  two  ways,  it  was  still  evident 
that  the  overall  tone  of  the  responses  was  quite  negative.  Three  of  four  respondents  who 
addressed  the  effectiveness  of  communications  asserted  that  no  cmnmunication  existed 
between  AMMOC  and  its  customer.  Of  the  other  responses  which  addressed  the  nteans 
of  communicating  with  customers,  most  were  seemingly  very  weak,  and  some  were 
qualified  by  the  respondent  as  insufficient.  For  example,  one  respondent  said,  "officiaily 
tile  communications  go  through  major  command  channels....  Unfortunately,  we  call 
directly  to  the  field...."  The  implication  was  that  the  official  communication  channels  were 
ineffective.  Three  respondents  indicated  that  Utilization  and  Training  Workshops 
(U&TWs)  were  a  primary  mode  of  communication  with  customers.  Recall  from  Chapter 
n  that  only  two  U&TWs  have  been  conducted  since  1987.  Three  respondents  stated  that 
the  communication  primarily  resulted  from  informal  interaction  with  students  ot  through 
formal  student  critiques.  And  finally,  some  respondents  indicated  that  primary 
communications  transpired  through  occasional  guest  speakers  from  the  field,  a  relatively 
weak  form  of  communication. 

There  seemed  to  be  no  common  theme  among  those  who  responded  by  identifying 
the  means  by  which  AMMOC  communicates  with  its  customers.  This  suggests  that  there 
probably  is  not  an  effective  line  of  communication  between  AMMOC  and  its  customers. 
This  idea  is  consistent  with  the  responses  of  those  who  interpreted  the  questions  in  terms 
of  communication  effectiveness.  The  responses  to  this  question  indicate  that  there 
probably  are  significant  opportunities  to  improve  customer  communications. 
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Feedback 

To  gather  more  information  concerning  the  effectiveness  of  communications 
between  AMMOC  and  its  customers,  some  supposedly  established  methods  for  soliciting 
customer  feedback  were  presented  in  the  second  round.  The  following  feedback 
mechanisms  defined  in  AETCR  52-12 ,  Training  quality  assessment,  were  presented  to 
the  instructors  in  die  second  round: 

1.  Training  Quality  Reports  (TQRs) 

2.  Field  Evaluation  Questionnaires/Training  Evaluation  Reports 

3.  Customer  Service  Information  Line  (CSBL) 

4.  Field  Interviews 

5.  Occupational  Survey  Reports 

6.  Subject  Matter  Expert  Reports 

7.  Graduate  Assessment  Surveys 

While  the  responses  were  mixed,  overall  instructors  rated  these  mechanism  as 
fairly  ineffective  (Appendix  G,  Question  B.3).  In  fact,  only  two  of  these  feedback 
mechanisms,  field  interviews  and  graduate  assessment  surveys,  were  rated  as  effective  by 
at  least  50  percent  of  the  respondents.  Furthermore,  four  of  the  five  remaining  options 
received  four  or  less  favorable  responses  from  the  seventeen  instructors. 

Through  further  research  presented  in  Chapter  II,  the  researchers  learned  that  none 
of  these  methodls  of  customer  feedback  were  presenting  AMMOC  with  significant 
feedback:  Graduate  Assessment  Surveys  were  not  used  at  all  for  AMMOC;  no  TQRs  had 
been  received  from  the  field  in  several  years;  only  two  occupational  surveys  had  been 
conducted  in  the  last  14  years;  CSIL  was  unpublicized  to  AMMOC  graduates;  and  field 
surveys  were  conducted  only  in  small  numbers.  Given  these  facts  along  with  the 
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responses  to  the  previous  question,  it  is  evident  that  communication  between  AMMCK! 
and  its  customers  is  quite  minimal. 

Communications 

The  following  questions  were  asked  in  round  one  concerning  the  communications 
between  AMMOC  and  the  other  organizations  associated  with  AMMCXT  training  and 
training  development 

a.  In  your  opinion,  how  effective  are  the  lines  of  communication 
between  AMMOC  and  the  various  organizations  (Operational  MAJCOMs, 
MPC,  AETC,  Air  Staff,  etc)  involved  in  the  system  of  training  and  training 
development? 

b.  How  can  communications  be  improved? 

The  first  question  was  intended  to  determine  the  instructors'  perceptions  regarding 
the  effectiveness  of  communication  with  organizations  that  should  contribute  to 
AMMOC's  instructional  efforts.  The  responses  indicated  an  overwhelming  perception  by 
the  instructors  that  these  c(»nmunications  are  inefiective.  Fifteen  of  sixteen  respondents 
indicted  some  level  of  negativity  and  twelve  of  sixteen  indicated  strong  negativity  by 
including  comments  such  as,  ”100%  ineffective,"  "Not  very  good,"  "Marginally  effective," 
and  "Very  limited  in  formal  reviews  and  nonexistent  in  informal  data  gathering."  From 
these  responses  it  was  evident  that  communicatitxi  between  AMMOC  and  the  agencies 
that  support  AMMOC  is  ineffective  and  provides  a  significant  opportunity  for 
improvemoit 

The  second  part  of  the  question  solicited  ideas  from  AMM(X:  instructOTS  on  how 
the  communications  could  be  improved.  The  respondents’  ideas  were  compiled  and 
presented  back  to  the  instructors  in  the  second  round  (Appendix  G,  (^estion  B.4). 
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Most  of  the  ideas  submitted  for  improving  AMMOC’s  communication  with  outside 
organizations  received  strong  support  from  the  instrucmrs.  The  ideas  which  received 
strong  support  are  presented  below  in  Table  4-3. 

Table  4>3.  Strongly  Supponed  Ideas  f^  In:^)roving  Communications 

1.  Provide  AMMOC  access  to  electronic-mail  for  correspondence  with  organizations 
throughout  the  Air  Force. 

2.  Permit  and  fimd  instructor  TDYs  to  operational  wings  to  maintain  AMMOC’s 
currency  with  field  operations  and  policies. 

3.  Place  AMMOC  on  distribution  for  changes  in  policies  and  regulations. 

4.  Include  AMMOC  in  pertinent  message  traffic  for  informational  purposes  to  maintain 
AMMOCs  currency  on  unit  level  maintenance  policies  and  practices. 

5.  Establish  points  of  contact  within  MAJCOMs,  research  centers  and  depots.  These 
points  of  contact  could  forward  information  to  AMMOC. 

6.  Conduct  surveys  of  former  students  and  commanders. 

7.  Provide  a  system  through  which  the  various  organizations  may  provide  inputs  or 
suggestions  pertaining  to  AMMOCs  training  on  an  ongoing  basis,  not  only  at  a  U&TW 
ccmference. 

Two  of  the  improvement  ideas  received  unanimous  support.  First,  instructors 
believed  Temporary  Duties  (TDYs)  at  operational  wings  would  help  maintain  AMMOCs 
currency  with  field  operaticms.  Second,  instructors  believe  AMMOC  should  be  on 
distribution  fcv  changes  in  policies  and  regulations.  These  two  ideas  were  also  presented 
as  suggestions  for  improving  AMMOCs  adaptability  to  change  in  the  next  section  of  the 
second  round  questionnaire. 

Item  seven  in  Table  4-3  suggests  that  a  system  be  implemented  that  allows  other 
organizations  to  easily  access  AMMOC  regarding  training  issues.  Item  one,  the 


suggestion  to  provide  e-mail  access  for  AMMOC,  would  certainly  serve  as  an  avenue  for 
soliciting  feedback  concerning  training  which  was  addressed  in  item  seven.  As  expected, 
because  these  items  are  related,  they  received  similar  levels  of  support  among  the 
instructors.  Likewise,  item  six  concerning  the  survey  of  former  students  also  received 
similar  levels  of  support 

It  is  interesting  to  note  that  several  of  the  communication  improvement  ideas  could 
be  initiated  internally  by  AMMOC.  AMMOC  can  place  itself  on  distribution  for  relevant 
regulations  and  changes,  and  can  identify  points  of  contact  within  MAJCOMs. 
Furthermore,  while  the  training  group  staff  function  of  Evaluation  is  primarily  responsible 
for  soliciting  feedback  from  graduates,  we  know  of  no  rules  that  preclude  AMMOC  from 
also  soliciting  customer  feedback.  However,  as  discussed  later,  the  question  arises  as  to 
whether  instructors  have  time  for  such  initiatives,  and  whether  organizational  structure 
inhibits  the  open  commuiucations  necessary  to  initiate  direct  communications  with  other 
orgaiuzations. 

From  the  responses  addressed  in  this  and  previous  sections,  there  is  strong 
evidence  to  suggest  that  communication  between  AMMOC,  its  external  customers,  and 
the  training  support  system,  is  relatively  ineffective  and  a  strong  candidate  for 
improvemenL 

Adaptability  to  Change 

Four  first-round  questions  concerned  the  training  and  training  development 
system's  ability  to  adapt  training  to  changes  in  maintenance  and  management  concepts  in 
the  field.  Two  aspects  of  adaptability  were  emphasized;  one  on  the  system's 
responsiveness  (timeliness)  to  change  and  the  other  on  the  system 's  flexibility  in 
accommodating  change  (relative  ease). 
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Responsiveness.  The  two  questions  relating  to  responsiveness  were  as  follows: 


a.  How  responsive  is  the  present  system  of  training  and  training 
development  to  incorporating  changes  in  maintenance  and  management 
concepts  employed  in  the  field? 

b.  How  can  this  system  become  more  responsive? 

Eleven  of  the  fifteen  responses  (one  no  response)  were  negative  in  nature.  The 
select  answers  presented  in  Table  4-4  capture  the  essence  of  the  negative  responses. 


Table  4-4.  Representative  Responses  Relating  to  Training  System  Responsiveness. 


a.  How  responsive  is  the  present  system  of  training  and  training 
development  to  incorporating  changes  in  maintenance  and  management 
concepts  employed  in  the  field? 

1.  Slow,  because  of  the  AETC  layers  of  approval  to  POI  [Plan  of  Instruction]. 

2.  Not  responsive  at  all.  Too  much  time/approval  needed  to  make  changes. 
Instructors  learn  of  changes  in  the  field  from  STUDENTS! 

3.  Not  very  responsive  since  we  are  tied  to  the  CTS  (course  training  standard) 
identified  by  those  above  AMM(X  (ex-  Air  Staff,  MAJCOMs). 

4.  Training  development  is  primarily  the  individual  instructor's  responsibility. 
Due  to  instructors  in  class  without  extended  t^aks,  the  development  system  is  almost 
nonexistent 


The  negative  responses  tended  to  focus  on  the  constraints  of  the  course  training 
standard  (CTS),  insufficient  time  to  conduct  course  development,  and  AMMOC’s 
unawareness  of  impending  changes  in  the  field.  Conversely,  the  two  positive  responses 
indicated  that  instructors  do  have  some  control  over  the  training  development  process. 
However,  we  know  from  Chapter  II  that  this  development  must  occur  within  the  confines 
of  the  CTS.  Furthermore,  one  of  the  positive  responses  stipulated  that  the  system  was 
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responsive,  but  indicated  that  this  responsiveness  was  contingent  on  AMMOC's  awareness 
of  required  changes,  and  on  the  opportunity  to  incorporate  such  changes  into  lesson  plans. 

The  second  part  of  the  question,  “How  can  this  system  become  more  responsive?,” 
yielded  several  ideas  which  were  subsequently  presented  in  the  second  round 
quesdonnaire.  Unfortunately,  when  these  ideas  were  presented  back  to  the  instmctors,  the 
questionnaire  item  was  flawed  Instead  of  referring  to  “responsiveness,”  the  question 
addressed  “flexibility, "  which  was  also  the  topic  of  the  next  second  round  question  (See 
Appendix  G,  Question  B.5.a).  Because  of  the  similarity  between  the  concepts  of 
“responsiveness”  and  “flexibility,”  we  believe  that  the  following  improvement  ideas 
displayed  in  Table  4-5,  which  received  strong  support  in  the  second  round,  are  still  valid 
as  improvement  ideas. 

Table  4-5:  Ideas  for  Improving  Training  System  Responsiveness 

1.  Allow  instructors  to  go  TOY  and  then  update  lesson  plans. 

2.  Provide  instructors  time  to  conduct  course  development. 

3.  Establish  a  cadre  of  personnel  for  course  development  to  relieve  instmctors  of  the 
burden  of  both  developing  and  providing  training. 

4.  Establish  MAJCOM  and  Air  Staff  points  of  contact  for  AMMOC  which  include 
office  symbols,  names  and  telephone  numbers. 

5.  Empower  AMMOC  staff  to  change  the  Course  Training  Standard  (CTS)  when 
changes  in  maintenance  policies  and  practices  warrant  it 

6.  Include  AMMOC  in  pertinent  message  traffic  for  informational  purposes  to 
maintain  AMMOC's  cmrency  on  unit  level  maintenance  policies  and  practices. 

7.  Improve  communication  between  AMMOC,  and  USAF  and  MAJCOMs  regarding 
impending  changes  to  maintenance  policies  and  practices. 
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As  shown  in  Table  4-5,  one  idea  was  to  empower  AMMCX!  to  change  the  CTS 
when  changes  in  the  field  warrant  it.  With  14  of  17  respondents  in  favor  of  the  idea,  the 
support  was  not  unanimous,  but  nonetheless  quite  high.  The  support  of  this  idea  was  also 
ccmsistent  with  some  complaints  during  the  first  round  that  AMMOC’s  unresponsiveness 
is  related  to  constraints  imposed  by  the  CTS.  Recall  from  Chapter  II  that  the  CTS  is 
essentially  a  contract  between  AMMOC  and  its  customers,  and  establishes  AMMOC’s 
course  objectives.  It  was  noted  in  Chapter  II  that  the  forum  for  updating  the  CTS  is  the 
Utilization  and  Training  Workshop  (U&TW),  which  is  a  relatively  infrequent  event 

Support  of  other  ideas  were  also  consistent  with  previous  questionnaire  items  on 
communications.  Once  again,  instructor  TDYs  to  operational  wings,  the  establishment  of 
points  of  contacts  at  major  commands  and  Air  Staff,  and  AMMOC’s  inclusion  in  pertinent 
ctxnmunications  concerning  maintenance  policies  all  received  strong  support  in  this 
section. 

Consistent  with  the  first  round  responses,  instructors  again  identified  opportunity 
for  course  development  as  a  factor  in  improving  responsiveness.  For  example,  providing 
more  time  to  instructors  for  course  development  received  unanimous  suppon  from  the 
instructors  as  a  means  for  improving  responsiveness.  Finally,  the  ideas  of  more  manpower 
for  course  development  also  received  strong  support  Recall  from  Chapter  II  that  course 
design  calls  for  eight  hour  training  days.  If  instructor  manning  is  limited  and  if  instructors 
spend  most  of  their  time  in  the  classroom,  it  certainly  makes  sense  that  manpower  and 
time  would  be  factors  in  AMMOC’s  responsiveness  in  incorporating  changes  to  the 
course.  This  point  is  underscored  in  Section  C,  question  4,  of  the  first  round 
questionnaire  which  specifically  addresses  the  relationship  between  course  design  and 
oppmtunity  for  course  development.  This  idea  was  expressed  as  follows  by  one  first 
round  respondent. 
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Presently  the  course  relies  entirely  upon  the  instructors  to  write,  research,  and 
update  the  material.  There  are  no  d«licated  people  or  enough  extra  instructors  to 
allow  for  good  research  and  writing  time.  We  need  one  or  the  other. 

The  issue  of  oppwtunity  for  course  development  was  raised  again  and  again  throughout 

the  questionnaire  sections  concerning  scheduling,  class  hours,  time  for  research,  and 

course  administration. 

Flexibility.  The  next  two  questions  from  the  first  round  concerning  adaptability 
to  change  addressed  the  flexibility  of  the  system. 

c.  How  flexible  is  the  present  system  of  training  and  training 
development  to  incorporating  changes  in  maintenance  and  management 
concepts  employed  in  the  field? 

d.  How  can  this  system  become  more  flexible? 

As  stated  earlier,  these  questions  were  asked  to  determine  how  easily  changes  are 
incorporated.  Although  four  of  the  sixteen  responses  were  positive,  the  vast  majority 
were  quite  negative.  Once  again  common  themes  throughout  the  responses  concerned  the 
constraints  of  the  CTS,  poor  communications,  and  insufficient  time  to  incorporate 
authorized  changes.  It  was  also  apparent  that  instructors  were  allowed  to  change  their 
particular  instructional  blocks  without  much  trouble  provided  they  were  aware  changes 
were  needed,  provided  they  had  ample  time  to  incorporate  changes,  and  provided  the 
changes  were  consistent  with  the  Course  Training  Standard  (CTS). 

In  the  second  question  above,  question  d,  instructors  were  asked  to  suggest  ways 
of  improving  the  flexibility  of  the  training  system.  Some  suggestions  were  similar  to  the 
those  for  improving  the  responsiveness  of  the  training  system.  For  example,  it  was 
suggested  that  instructor  be  given  time  to  travel  to  operational  wings  to  research  and 
incorporate  new  developments  into  their  lesson  plans.  Some  responses  related  to 
empowerment  of  instructors  to  have  more  control  over  class  schedules,  class  sizes,  and 
course  content.  As  in  previous  cases,  these  ideas  were  presented  in  the  second  round 
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questionnaire  to  detennine  the  level  of  support  among  all  instructors.  The  ideas  supported 
are  presented  in  Table  4-6. 


Table  4^.  Meas  for  Improving  the  Rexibility  of  the  Training  System 


«  1.  Send  instructors  TDY  to  operational  units  to  increase  their  awareness  of  current 

policies  and  practices. 

2  Include  AMMOC  in  pertinent  message  traffic  for  informational  purposes  to  maintain 
AMMOC's  cunency  on  unit  level  maintenance  policies  and  practices. 

3.  Give  instructors  inputs  on  planning  course  schedules,  class  sizes,  and  down  times. 

4.  Distinguish  course  development  responsibilities  between  AMMOC  instructors  and 
Training  Development  Element  (TDE)  personnel. 


As  expected,  the  second  round  questionnaire  items  concerning  instructor  TDYs 
and  commtmications  received  strong  support,  just  as  in  other  sections  of  the  questionnaire. 
Furthermore,  the  questionnaire  item  addressing  empowerment  of  instructors  to  influence 
course  schedules,  class  sizes,  and  instructor  "downtimes”  also  received  strong  support 
which  is  consistent  with  repeated  occurrences  throughout  the  questionnaires  where 
instructors  asserted  that  more  control  over  scheduling  was  needed  and  that  insufficient 
time  for  course  development  was  available  to  instructors. 

The  last  improvement  issue  concerning  flexibility  dealt  with  the  delineation  of 
course  development  responsibilities  between  AMMOC  instructors  and  Training 
Development  Element  (TDE)  personnel.  Apparently  their  is  little  guidance  concerning 
where  their  responsibilities  are  split  This  is  one  area  were  additional  guidance  for  the 
process  may  be  required. 
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Finally,  one  item  in  this  section  addressed  management  philosophies.  After  further 
analysis,  the  researchers  considered  this  question  as  leading  and  did  not  include  it  in  the 
formulation  of  conclusions  regarding  flexibility  of  the  training  system. 

Instructor  Performance  Evaluation 

Three  questions  were  asked  concerning  instructor  performance  evaluation.  They 

were: 

a.  How  is  the  performance  of  individual  instructors  measured  and 
evaluated? 

b.  What  are  the  strengths  and  weaknesses  of  these  measures? 

c  How  can  AMMOC  instructor  performance  evaluations  be 
improved? 

The  intent  of  this  question  was  to  determine  if  there  were  meaningful  forms  of 
instructor  evaluation,  and  if  there  were  forms  of  instructor  evaluation  which  were  contrary 
to  training  effectiveness.  Unfortunately,  not  many  powerful  ideas  were  generated  by  these 
questions. 

Nearly  every  respondent  mentioned  the  AETC  Form  281,  Instructor  Evaluation 
Checklist,  used  by  instructor  supervisors  to  evaluate  instructor  classroom  performance. 
These  periodic  evaluations  entail  the  supervisors'  observation  and  documentation  of  the 
instructor's  performance  in  the  classroom.  Most  of  the  comnoents  were  fairly  neutral. 
However,  two  instructors  did  state  that  these  types  of  evaluations  were  worthless  and  tliat 
no  real  instructor  assessment  takes  place.  Several  respondents  identified  indirect  means  of 
evaluation  such  as  student  critiques  and  test  score  averages.  Only  one  individual 
mentioned  the  content  of  instructors'  lesson  plans  as  a  performance  measure. 

The  second  part  of  the  question  asked  the  for  strengths  and  weaknesses  of 
performance  measures  employed  by  AMMOC.  Only  four  respondents  cited  strengths  of 
the  periodic  supervisor  evaluations.  The  strengths  were  that  these  evaluations  provide 
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immediate  constructive  feedback  to  instructors  and  that  they  are  reliable.  However,  ten 
respondents  identified  weaknesses  related  to  the  periodic  evaluations.  These  included 
personal  bias  of  the  evaluator,  lack  of  emphasis  on  course  content,  evaluator  unfamiliarity 
with  instructional  material,  and  the  infrequency  of  the  evaluations.  From  the  first  round 
responses,  it  appears  that  there  may  be  a  significant  level  of  dissatisfaction  among 
instructors  concerning  the  periodic  evaluations.  Because  we  were  concerned  with  the 
length  of  the  second  round  questionnaire,  these  strengths  and  weaknesses  were  not  carried 
forward  to  the  second  round,  and  as  a  result  we  cannot  say  for  certain  what  level  of 
dissatisfaction  exists.  Instructor  evaluation  would  be  a  good  topic  for  future  research. 

The  third  part  of  this  question  asked  for  ideas  for  improving  instructor 
performance.  The  suggestions  pertained  not  only  to  the  periodic  evaluations,  but  also  to 
some  of  the  indirect  measures  of  instructor  performance  as  well.  Three  questions  from 
this  areas  are  presented  in  Table  4-7. 

Table  4-7,  Ideas  for  Improving  Instructor  Performance  Measures 

1.  AETC  Form  281  should  place  more  emphasis  on  lesson  content  than  instructor 
presentations  skills. 

2.  Implement  a  more  descriptive  AETC  Form  281  giving  more  specific  expectations 
of  performance. 

3.  Conduct  infernal  evaluations  in  addition  to  formal  evaluations. 


Item  1,  which  suggests  more  emphasis  on  lesson  content,  was  supported  by  less 
than  half  of  the  respondents.  This  is  surprising  because  in  the  first  round  questionnaire 
many  of  the  weaknesses  relating  to  periodic  instructor  evaluations  focused  on  the  AETC 
evaluation  form's  emphasis  on  presentation  skills  as  opposed  to  lesson  content.  The  last 
two  items  in  Table  4-6  did  receive  strong  support  with  14  of  17  respondents  either 
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agreeing  or  strongly  agreeing  with  the  ideas.  Five  of  the  six  remaining  items  in  this  section 
of  the  questionnaire  did  not  receive  strong  support.  One  of  the  remaining  items  that  was 
supported  was  determined  to  be  leading  during  the  analysis  of  these  findings,  and  therefore 
did  not  yield  any  meaningful  information. 

Though  there  was  some  dissatisfaction  expressed  with  the  evaluation  process,  few 
of  the  ideas  for  improving  instructor  performance  evaluation  received  strong  support  As 
stated  earlier,  instructor  performance  evaluation  would  be  a  good  topic  for  future 
research. 

AMMOC  Performance  Evaluation 

The  previous  questions  focused  on  individual  performance  measures.  Conversely, 
the  following  questions  concern  the  performance  of  AMMOC  as  a  whole. 

a.  How  is  the  performance  of  the  AMMO  Course  measured  and 

evaluated? 

b.  What  are  the  strengths  and  weaknesses  of  these  measures? 

c.  How  should  the  AMMO  Course  be  evaluated? 

There  were  several  methods  identified  for  measuring  and  evaluating  AMMOC 
performance.  Three  were  indirect  measures  of  performance  such  as  student  course 
critiques,  test  averages,  percentage  of  students  graduating,  and  number  of  students 
graduating.  Others  suggested  AMMOC  performance  is  evaluated  by  feedback 
mechanisms  such  as  Training  Evaluation  Reports  and  field  surveys.  Four  instructors 
stated  that  they  knew  of  no  measures  of  performance.  Three  instructors  did  not  respond. 
Overall,  it  was  evident  from  the  first  round  responses  that  there  was  little  awareness  of  any 
systematic  method  for  evaluating  AMMOC’s  overall  performance. 

The  second  part  of  the  question  asked  respondents  to  identify  strengths  and 
weaknesses  of  AMMOC's  performance  measures.  Only  one  instructor  identified  any 
strengths  with  the  performance  measures  they  had  identified  in  the  previous  pan  of  the 
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question.  However,  all  twelve  instructors  who  responded  did  identify  various  weaknesses 
with  present  performance  measures.  The  third  part  of  the  question  asked  for  instructors' 
ideas  for  improving  AMMOC's  performance  measures.  The  few  ideas  presented  in  the 
second  round  questionnaire  which  received  the  strongest  support  are  shown  in  Table  4-8 
below. 


Table  4-8.  Ideas  for  Improving  AMMOC's  Performance  Measures 

1.  Get  an  evaluation  "expert"  to  develop  an  evaluation  plan. 

2.  Annual  questionnaire  to  field  commanders  about  how  our  students  are  doing  and 
what  we  can  do  to  help  them  out. 

3.  Evaluate  graduates'  job  performance  in  the  field. 


14  of  17  instructors  agreed  that  AMMOC  should  have  an  expert  develop  an 
evaluation  plan  for  AMMOC.  This  idea  is  consistent  with  the  previous  observation  that 
there  seemed  to  be  little  awareness  among  instructors  of  an  existing  systematic  method  of 
evaluating  AMMOC.  The  other  two  ideas  concerned  the  solicitation  of  feedback  from  the 
field.  The  instructors'  support  of  these  ideas  are  consistent  with  earlier  responses  to  earlier 
questionnaire  items  that  suggested  a  lack  of  effective  communications  with  customers. 

The  concept  of  course  evaluation  is  very  complex.  Whether  or  not  some  of  these 
measures  such  as  student  throughput  and  test  scores  are  consistent  with  promoting  quality 
training  is  debatable.  Course  evaluation  is  also  an  excellent  topic  for  future  research. 

Empowemrient 

We  asked  a  variety  of  questions  concerning  empowerment  in  the  first  round 

a.  As  an  instructor,  what  authority  should  you  have  which  you 
currently  lack?  Why  should  you  have  this  authority  (what  advantage  is 
gained)? 
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b.  What  restrictions  or  requirements  have  been  placed  on  AMMOC 
as  a  whole,  by  regulations  or  any  other  guidance,  that  make  training  more 
difficult? 

c.  Can  the  constraints  in  6b.  above  be  removed  without  adversely 
affecting  the  quality  of  training? 

Two  of  sixteen  respondents  indicated  they  had  all  the  authority  they  needed,  and 
four  did  not  respond.  However,  the  remaining  responses  raised  many  of  the  same  issues 
addressed  in  previous  questions  such  as  class  schedules,  time  for  coiu^e  development,  and 
modification  of  the  CTS.  Another  issue  addressed  concerned  the  instructor's  authority  to 
control  the  length  of  the  training  day,  an  issue  which  received  much  attention  in  the  first 
round  questionnaire  section  on  that  subject 

Because  most  of  these  issues  were  specifically  addressed  in  other  sections  of  the 
questionnaire,  the  section  on  empowerment  was  eliminated  from  the  second  round. 
However,  each  response  discussed  above  was  presented  in  the  second  round  questionnaire 
in  the  section  most  pertinent  Responses  which  did  not  neatly  fit  into  an  existing 
questionnaire  category  were  consolidated  and  presented  in  the  General  Issues  section  at 
the  end  of  the  questionnaire. 

The  second  part  of  the  question  sought  to  identify  external  constraints  placed  on 
AMMOC.  In  general,  the  responses  were  varied,  focusing  on  issues  such  as  manpower, 
course  content,  course  length,  instructor  TDYs,  student  throughput,  and  U&TWs. 
Additionally,  a  few  issues  concerning  constraints  internal  to  AMMOC  were  addressed; 
however,  these  issues  will  not  be  discussed  because  they  are  beyond  the  scope  of  this 
thesis  which  is  on  external  factors  affecting  AMMOC.  As  was  the  case  with  the  previous 
question,  the  relevant  responses  were  presented  in  the  second  round  questionnaire  under  a 
topic  other  than  empowerment.  The  third  part  of  the  empowerment  question  addressed 
whether  the  constraints  identified  in  part  two  of  the  question  could  be  eliminated  without 
adversely  affecting  the  quality  of  training.  Every  respondent  who  responded  to  the  second 
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part  of  the  question  asserted  that  the  constraints  could  be  removed  without  degrading  the 
quality  of  training. 

Critique  Program 

Two  questions  were  asked  during  the  first  round  pertaining  to  the  student  critique 
program. 

a.  What  are  the  strengths  and  weaknesses  of  the  student  critique 
program? 

b.  If  you  could  improve  the  usefulness  of  the  student  critique 
program,  what  would  you  change,  and  how? 

The  purpose  of  this  question  was  to  identify  opportunities  to  improve  support  and 
guidance  beyond  AMMOC’s  control.  While  many  strengths  and  weaknesses  were 
identified  with  the  program,  and  while  several  suggestions  were  offered  for  improving  the 
program,  nearly  all  were  within  AMMOC's  control.  These  items  were  included  in  the 
second  round  as  a  courtesy  to  AMMOC'i  supervisors  under  the  premise  that  this  feedback 
might  be  of  use  to  them  Because  AMMOC's  internal  affairs  are  not  the  subject  of  this 
thesis,  these  issues  are  not  fiuther  addressed.  It  should  be  noted  that  several  of  these 
second  round  items  have  been  found  to  be  leading  after  further  review  and  may  not  be 
useful. 

Course  Objective  Development 

We  should  point  out  that  the  terms  "CTS  objectives"  and  "course  objectives"  are 
used  synonymously  in  our  discussion.  A  couple  of  respondents  indicated  that  these  terms 
are  not  the  same.  Usually,  course  objectives  arc  developed  from  CTS  task  and  knowledge 
statements  (Department  of  the  Air  Force,  1994c:  50).  However,  in  AMMOC's  case,  the 
training  objectives  are  taken  verbatim  from  task  and  knowledge  statements  in  the  CTS. 
Therefore,  the  course  objectives  and  CTS  elements  are  the  same  for  AMMCXI!. 
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Throughout  the  discussion  that  follows,  we  will  continue  to  use  the  terms  "course 
objectives"  and  "CTS  elements"  synonymously. 

The  purpose  of  this  section  was  to  determine  if  there  were  weaknesses  in  the 
current  course  objective  development  process,  and  to  identify  potential  improvement 
opportunities.  The  questions  asked  in  the  first  round  were: 

a.  What  are  the  strengths  and  weaknesses  of  the  current  method  of 
developing  course  objectives  [CTS  elements]? 

b.  Do  you  know  of  a  better  method  for  developing  course  objectives 
[CTS  elements]? 

Four  respondents  did  not  respond  to  the  first  question.  Nearly  all  the  comments 
concerned  weaknesses  of  course  objective  development  For  example,  four  respondents 
commented  on  the  nonresponsiveness  of  the  CTS  to  changes  in  the  field.  Three 
contended  that  CTS  items  were  vague,  and  that  it  was  difficult  to  determine  the  intent  of 
the  CTS'  authors.  Two  individuals  addressed  the  actual  development  process,  inferring 
that  there  was  not  enough  field-level  involvement  developing  CTS  elements.  Finally,  two 
comments  were  positive.  One  respondent  contended  that  CTS  elements  were  precise  and 
clearly  defined,  and  another  commented  that  once  the  CTS  is  changed,  the  course 
responds  relatively  quickly.  All  of  these  ideas  were  presented  in  the  second  round 
questionnaire  to  determine  if  there  was  agreement  concerning  the  strengths  and 
weaknesses  of  course  objective  development 

Of  the  seven  weaknesses  presented,  six  received  fairly  strong  support.  Two  of  the 
weaknesses  concerned  the  vagueness  of  course  objectives,  and  the  difficulty  associated 
with  interpreting  the  meaning  of  CTS  elements.  Respondents  generally  agreed  that  both 
were  weaknesses.  However,  a  related  questionnaire  item  that  characterized  course 
objectives  as  precise  and  clearly  defined  also  received  die  agreement  of  about  one  half  of 
the  respondents.  The  reason  for  this  apparent  disparity  is  not  clear.  The  two  items  that 
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were  presented  as  weaknesses  were  accompanied  by  examples  to  clarify  their  meanings, 
and  did  receive  similar  levels  of  agreement  among  respondents.  We  believe  these  items 
were  the  more  reliable,  and  conclude  that  instructors  do  view  CIS  items  as  vague.  This 
conclusion  is  supported  by  another  second  round  questionnaire  item  that  suggests  each 
CTS  item  be  accompartied  by  a  narrative  statement  which  explains  the  intent  of  the  CTS 
authors. 

The  two  items  that  characterized  the  CTS  as  nonresponsive  to  changes  in  the  field 
were  also  agreed  upon  by  the  majority  of  instructOTS.  Finally,  the  respondents  expressed 
that  the  relatively  low  level  of  proficiency  required  by  the  course  objectives  was  also  a 
weakness. 

The  second  part  of  the  question  on  course  objective  development  asked  instructors 
to  suggest  a  better  method  of  developing  course  objectives.  Seven  participants  did  not 
respond  to  this  item.  The  relevant  responses  to  this  question  were  presented  in  the  second 
round  questionnaire.  These  items  are  shown  in  the  table  below. 

Table  4-9.  Ideas  for  Improving  Course  Objective  Development 

1.  Develop  an  "on-line"  system  for  developing  and  maintaining  course  objectives  that 
is  accessible  by  AMMOC,  Air  Staff,  and  the  major  commands. 

2.  AMMOC  should  be  empowered  to  develop  its  own  course  objectives  with  the 
oversight  of  MAJCOMs  and  Air  Staff. 

3.  AMMOC  should  be  empowered  to  develop  course  objectives,  and  represent  the 
various  MAJCOMs  in  the  process. 

4.  Course  objectives  should  be  developed  with  the  involvement  of  maintenance 
leaders  from  operational  units  and  MAJCOM  staffs  who  range  ftom  company  to  field- 
grade  ranks.  The  development  process  should  include  a  review  of  current  AMMOC 
lesson  plans. 

5.  Each  CTS  element  should  have  a  short  nanative  statement  outlining  what  aspects 
of  the  topic  should  be  taught.  This  will  convey  the  authors'  intents  pertaining  to  each 
CTS  element 
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The  idea  of  an  "on-line"  system  for  developing  and  maintaining  course  objectives 
receive  support  ficom  15  of  17  respondents.  Two  items  were  presented  concerning  the 
empowerment  of  AMMOC  to  develop  course  objectives.  The  item  that  suggested 
AMMOC  independently  develop  course  objectives  did  not  receive  very  strong  support. 
However,  the  idea  that  AMMOC  should  develop  course  objectives  with  the  oversight  of 
the  major  commands  received  the  support  of  14  of  17  instructors.  The  remaining  two 
items  showed  that  instmctors  believed  that  CTS  objectives  should  be  developed  with  the 
involvement  of  maintenance  personnel  from  the  Held,  and  that  CTS  objectives  should  be 
accompanied  by  a  short  narrative  statement  that  conveys  the  intentions  of  the  CTS 
developers.  One  additional  question  which  received  strong  support  in  the  second  round 
questionnaire  is  not  discussed  here  because  the  researchers  determined  that  it  was  leading 
and  therefore  not  useful. 

Course  Control  Documents 

The  purpose  of  this  section  was  to  evaluate  course  control  documents  such  as 
course  training  standards,  course  charts,  and  plans  of  instruction  to  determine  if  there  are 
opportunities  to  improve  these  documents.  The  questions  asked  during  the  first  round 
were, 

a.  What  are  the  strengths  and  weaknesses  of  course  control 
documents? 

b.  If  you  could  improve  the  usefulness  of  the  course  control  documents 
by  changing  current  regulations  or  guidance,  what  would  you  change,  and 
how? 

The  responses  to  these  questions  were  similar  to  those  to  the  previous  questions 
concerning  course  objective  development  Fot  example,  respondents  again  commented 
that  CTS  statement  were  too  vague,  that  course  control  documents  quickly  become 
outdated,  and  that  no  master  plan  integrates  the  various  course  control  documents.  Two 
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respondents  did  identify  a  few  positive  aspects  of  course  control  documents  such  as 
training  standardization. 

The  second  part  of  the  question  solicited  ideas  for  improving  course  control 
documents.  Seven  instructors  did  not  respond  to  this  question  at  all.  Six  ideas  were 
suggested  during  round  one  and  were  presented  in  the  second  round  questionnaire.  Two 
of  the  six  ideas  did  not  receive  strong  support.  Three  that  received  unanimous  support 
were  determined  to  be  leading  and  were  not  of  any  use.  However,  their  was  one  item  that 
did  gamer  fairly  strong  support.  Eleven  of  fourteen  instmctors  agreed  that  AMMOC 
should  be  allowed  to  modify  course  control  documents.  This  was  consistent  with 
questionnaire  items  in  previous  sections  that  suggested  that  AMMOC  should  be 
authorized  to  modify  the  CTS.  However,  the  item  in  this  section  of  the  questionnaire  was 
not  as  strongly  supported.  It  was  shown  in  Chapter  n  that  AMMOC  instructors  already 
control  all  course  control  documents  except  for  the  CTS.  It  is  possible  that  some 
instructors  disagreed  with  this  statement  because  they  already  have  the  control  that  the 
questionnaire  suggests  that  they  do  not  have. 

Class  Hours,  Time  for  Research,  and  Course  Administration 

This  was  one  of  the  areas  where  there  was  strong  agreement  in  the  first  round.  As 
a  result,  follow  up  questions  were  not  asked  in  the  second  round.  Three  questions  were 
asked  concerning  class  hours,  time  for  research,  and  course  administration. 

a.  How  do  policies  governing  the  minimum  length  of  the  training  day 
affect  the  amount  of  time  you  have  for  research  and  course  administration? 

b.  What  do  you  think  should  be  the  policy  on  how  instructors  manage 
class  time?  Why? 

c.  What  should  be  the  minimum  number  of  classroom  hours  per  day? 
Should  there  be  a  standard  training  day? 
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In  response  to  the  first  question,  12  of  the  16  first  round  respondents  indicated  that 
the  policies  governing  the  minimum  length  of  the  training  day  did  affect  their  ability  to 
conduct  research  and/or  the  time  available  for  course  administration.  Representative 
responses  are  presented  in  Table  4-10. 


Table  4-10.  Select  First  Round  Responses  Concerning  the  Length  of  the  Ix-ining  Day. 


a.  How  do  policies  governing  the  minimum  length  of  the  training 

day  affect  the  amount  of  time  you  have  for  research  and  course 

administration? 

1.  The  training  day  is  too  long— 8  hours  of  teaching... degrades  instructor 
effectiveness  and  student  retention  rates. 

2.  There  is  no  time  for  anything  but  instructing. 

3.  It  hurts.  Unless  the  instructor  has  a  back  up  instructor  or  a  break  between 
classes  there  is  little  to  no  time  to  do  research  &  lesson  development 

4.  There  is  no  time  for  course  research  because  we  are  required  to  keep 
students  in  class  all  day,  even  if  material  has  been  covered. 

5.  Current  "policy"  is  unwritten.  Classroom  hours  are  07: 10  to  1620,  with 
lunch  from  1 1-1230.  This  current  "policy"  however  is  not  enforced,  so  most  instructors 
end  teaching  by  around  1430-1500.  Our  problem  in  having  time  to  research  is  back  to 
back  classes.  Most  blocks  of  instruction  have  only  one  primary  instructor,  some  have  a 
backup,  but  we  don't  use  the  backup  to  allow  the  primary  to  do  research,  only  for  leave 
&  other  special  events. 

6.  Currently,  we  are  required  to  teach  8  hrs  per  day.  This  means  the  instructor 
must  attend  to  course  development  over  lunch,  on  the  10  minute  break  each  hour,  or 
after  normal  duty  hours.  With  back  to  back  classes,  instructors  are  afforded  no 
"downtime"  for  course  development 


When  asked  what  they  thought  the  policy  on  class  time  management  should  be,  1 1 
of  the  16  respondents  specifically  indicated  the  instructors  should  have  direct  control  over 
the  time  spent  in  class.  An  additional  three  indicated  that  six  hours  of  instruction  would  be 
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appropriate  which  would  provide  the  instructor  two  hours  for  other  responsibilities  such 
as  research,  course  administration,  and  additional  duties.  It  would  also  give  students  two 
hours  for  student  self  study.  Table  4-11  presents  a  few  of  the  responses  that  summarized 
the  key  points  made  by  the  instructors  to  the  question,  what  do  you  think  should  be  the 
policy  on  how  instruct(»*s  manage  class  time? 

Table  4-11.  Select  First  Round  Responses  Concerning  Class  Time  Management 

b.  What  do  you  think  should  be  the  policy  on  how  instructors 

manage  class  time?  Why? 

1.  6  hrs  teaching,  2  hrs  course  work  and  development,  or  two  instructors  per 
block  and  dedicated  supervisors,  or  a  staff  of  training  development  &  course  writers. 

2.  If  we  want  to  make  it  a  reputable  course,  instructors  need  1-2  weeks 
between  classes  to  research;  or  develop  a  training  development  section  to  do  research, 
LP's  [Lesson  Plans],  Vis-aids.  (Not  our  current  TDB,  they  are  glorified  admin  clerks, 
an  added  layer,  with  no  added  value)  If  somebody  in  the  process  made  classes  6 
students  larger,  we  could  easily  handle  the  larger  classes,  separa^  new  classes  by  1 
week,  giving  instructors  time  to  train!,  research,  update,  improve!,  leam  a  little  about 
quality!  and  do  additional  duties. 

3.  Leave  to  the  discretion  of  the  instructor.  They  are  officers  and  should  be 
trusted  to  do  a  professional  job. 

In  discussing  the  minimum  number  of  classroom  hours  per  day,  13  of  the  16 
respondents  mentioned  six  hours  in  some  capacity  in  their  responses.  Seven  of  the 
respondents  set  the  minimum  at  six  hours,  three  set  six  hours  as  the  maximum,  and  three 
stated  the  standard  training  day  should  be  six  hours. 

Scheduling 

Scheduling  was  another  area  where  there  was  a  strong  majority  opinion  presented 
by  the  instructors  in  the  first  round.  Consequendy,  it  was  not  directly  addressed  again  in 
the  second  round.  The  first  round  question  was. 
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What  role,  if  any,  should  AM MOC  play  in  the  determination  of  class 
sizes,  steit  dates,  and  graduation  dates? 

Ten  of  the  sixteen  respondents  indicated  that  as  a  minimum  AMMOC  should  be 
consulted  in  some  form.  Two  of  the  ten  said  that  AMMOC  should  have  large  control  over 
class  size,  start  dates,  and  graduation  dates,  and  three  said  AMMOC  should  have  complete 
control  over  the  parameters.  Three  respondents  expressed  no  opinion,  and  the  other  three 
commented  on  specific  issues  such  as  class  size,  class  start  and  graduation  dates 
However,  these  individuals  did  not  specify  the  role  AMMOC  should  play.  From  the 
results,  we  found  strong  evidence  to  suggest  that  AMMOC's  involvement  in  student 
scheduling  was  limited,  and  that  AMMOC  instructors  desired  more  control  over 
scheduling. 

The  instructors  identified  several  benetits  that  could  be  achieved  if  AMMOC  were 
more  involved  in  scheduling.  First,  research  time  could  be  built  in  to  the  course  by 
scheduling  breaks  between  classes.  Second,  TOY  pay  to  students  could  be  saved  thrqugh 
smarter  scheduling  of  class  start  and  graduation  dates  that  shortened  students  period  of 
absence  frcxn  their  home  stations.  Examples  of  poor  scheduling  included  graduation  dates 
the  day  after  three  day  weekends,  and  a  class  start  date  which  was  scheduled  one  week 
before  a  two  week  Christmas  break.  Third,  a  few  of  the  instmctors  pointed  out  that 
instructor  leaves  and  shortfalls  could  be  managed  more  easily  if  they  were  more  involved 
in  class  scheduling. 

Administration 

One  question  was  asked  regarding  instmctor  administrative  responsibilities. 

Are  there  administrative  tasks  imposed  on  AMMOC  that  can  be 
streamlined  or  eliminated  without  adversely  affecting  training? 

Only  6  of  the  16  first  round  respondents  provided  substantial  responses.  Four  of 

the  six  identified  additional  duties  as  particularly  burdensome  and  difficult  to  accomplish- 
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a  result  of  the  limited  time  available  to  accomplish  them.  Since  the  issue  of  time  was 
addressed  in  other  areas  under  analysis,  Administration,  as  a  section,  was  omitted  from  the 
second  round. 

Other 

This  last  section  in  the  first  round  was  a  catch-all  for  other  areas  not  specifically 
addressed  elsewhere  in  the  questionnaires.  The  questions  were, 

1.  What  other  ideas  do  you  have  for  improving  the  support  provided 
to  AMMOC? 

2.  Any  other  comments. 

The  improvement  opportunities  and  improvement  ideas  generated  in  the  first  round 
questionnaire  that  did  not  fit  into  the  specific  sections  included  in  the  second  round,  were 
consolidated  in  a  General  Issues  section  in  the  second  round  for  the  instructors  to  review. 
Six  of  the  general  ideas  suggested  were  either  unanimously  supported  or  strongly 
supponed  by  the  instructors.  Two  of  these  six  questionnaire  items  were  determined  to  be 
leading  and  were  not  useful.  The  other  four  are  presented  in  table  below. 

Table  4*12.  General  Improvement  Ideas  Supported 

1.  Maintenance  officers  should  be  offered  an  incentive  to  become  AMMOC 
instructors  (e.g.,  choice  of  follow-on  assignment). 

2.  Facilities  should  be  improved. 

3.  Establish  a  library/research  area  for  students  and  staff. 

4.  Institute  job  related  exercises  pertaining  to  topics  such  as  mobility  and  battle  staffs. 


During  validation  of  the  first  round  questionnaire,  questions  penaining  to  facilities 
were  eliminated  to  shorten  the  length  of  the  questionnaire.  However,  from  the  responses 
to  item  2  in  the  table  above,  it  seems  that  there  is  a  high  level  of  interest  in  facility 
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improvement.  Facilities  would  be  a  good  topic  for  further  research.  Presented  below  are 
some  of  the  instructors  comments  concerning  facilities. 

a.  Our  facilities  are  terrible  the  temperatine  is  90  plus  inside  in  the  summer 
and  below  50  in  the  winter,  this  is  not  good  for  learning. 

b.  Facilities  are  terrible  considering  emphasis  on  "Quality."  There  are  new 
buildings  for  other  courses  coming  to  Sheppard  (i.e.  Lowry  courses),  but  no  plans 
for  AMMOC. 

Item  1  may  be  a  partial  solution  to  AMMOC's  apparent  manpower  shortage.  Given  that 
funding  is  available,  establishing  a  library  should  be  within  AMMOC's  control.  Finally,  the 
desire  to  institute  realistic  exercises  into  the  curriculum  is  consistent  with  the  frequently 
expressed  desire  for  AMMOC  to  exert  more  control  over  course  content  It  may  be 
possible  that  the  incorporation  of  such  exercises  may  be  permissible  under  AMMOC  s 
current  CTS. 

Barriers  to  Effective  Training 

As  explained  in  Chapter  in,  several  respondents  described  circumstances  in  the 
first  round  which  could  be  defined  as  barriers  to  effective  training.  We  thought  it  would 
be  useful  to  determine  if  AMMOC  instructors  did  in  fact  regard  these  conditions  as 
barriers,  and  if  they  did,  how  significant  the  barriers  were.  To  accomplish  this,  we 
presented  a  list  of  these  barriers  in  the  second  round  for  the  respondents  to  evaluate 
(Appendix  G,  pg.  G-4).  We  asked  the  respondents  to  first  express  whether  each  condition 
was  a  barrier.  Secondly,  we  asked  them  to  identify  the  ten  barriers  that  they  regarded  as 
the  most  significant. 

Unfortunately,  while  every  instructor  rated  their  agreement  or  disagreement  with 
each  of  the  barriers,  seven  did  not  identify  the  top  ten  barriers.  In  hind  sight,  the 
researchers  should  have  split  the  two  tasks  into  separate  sections  or  provided  a  statement 
at  the  end  of  the  question  reminding  the  respondents  to  go  back  and  rate  the  top  ten 
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barriers.  Despite  some  of  the  respondents’  apparent  confusion  in  completing  this  portion 
of  the  questionnaire,  useful  information  was  obtained.  Table  4-13  presents  the  items 
which  received  the  most  votes  as  barriers  to  effective  training.  The  table  identifies  the 
number  of  votes  each  item  received  as  a  top  barrier  and  the  number  of  respondents  that 
either  disagreed  or  strongly  disagreed  that  the  items  were  barriers  to  effective  training. 

The  respondents  who  cast  dissenting  votes  were  not  disagreeing  with  the  validity 
of  the  statement,  but  dissenting  with  the  fact  that  these  conditions  are  barriers  to  effective 
training.  All  of  these  barriers  are  consistent  with  responses  on  these  topics  in  other 
sections  of  the  questioimaire.  There  were  several  other  areas  that  were  strongly  supported 
as  barriers,  but  were  not  top  vote-getters  as  significant  barriers.  Table  4-14  includes  all 
the  barriers  for  which  two  or  less  dissenting  votes  were  cast. 

Table  4-13.  Instructor's  Top  Five  Barriers  to  Effective  Training 

1.  Maruiing  levels  and  back-to-back  classes  afford  instructors  inadequate  time  for 
conducting  research  and  course  development.  [10  votes]  [0  disagree] 

2.  Restrictions  on  maximum  class  size  and  student  throughput  result  in  continuous 
back-to-back  classes.  [7  votes]  [  3  disagree] 

3.  AMMOC  lacks  authority  to  modify  the  CTS.  [6  votes]  [3  disagree] 

4.  Process  for  changing  course  length  is  difficult.  [6  votes]  [1  disagree] 

5.  There  is  insufficient  funding  for  student  instructional  materials.  [6  votes] 

[0  disagree] 

Three  general  categories  were  not  presented  as  barriers  in  the  second  round 
questionnaire.  In  hind  sight,  communications,  scheduling,  and  facilities  should  have  been 
included.  Their  exclusion  was  a  result  of  an  oversight  on  the  pan  of  the  researchers.  If 
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included,  the  researchers  believe  the  three  would  have  been  rated  as  signiHcant  barriers. 
This  assumption  is  based  on  responses  in  both  the  first  and  second  round  questionnaire. 


Table  4-14.  Instructor  Identified  Barriers  with  Two  or  Fewer  Dissenting  Votes. 


1.  Manning  levels  and  back-to-back  classes  afford  instructors  inadequate  time  for 
conducting  research  and  course  development.  [10  votes] 

2.  Process  for  changing  course  length  is  difficult  [6  votes] 

3.  AMMOC  is  not  included  on  important  message  traffic  pertaining  to  maintenance 
policies  and  practices.  [5  votes] 

4.  Vaguely  worded  CTS  objectives  make  it  difficult  to  determine  the  intent  of  the  CTS 
authors.  [5  votes] 

5.  AMMOC  instructors  are  ftequently  pulled  from  the  course  to  fill  other  positions  in 
the  group.  [5  votes] 

6.  There  is  no  incentive  to  streamline  training  because  any  resources  saved  are  taken 
away.  [5  votes] 

7.  There  is  insufficient  funding  for  student  instmctional  materials.  [6  votes] 


Summary 

The  responses  from  both  the  first  and  second  round  of  the  Delphi  provided 
valuable  information  for  identifying  improvement  opportunities  for  the  guidance  and 
support  provided  to  AMMOC.  The  instructors  were  in  fact  significant  resources  for  this 
process.  The  management  in  the  training  system  that  supports  AMMOC  should  take 
special  note  of  the  responses  for  they  represent  the  voice  of  their  internal  customer.  The 
subjective  value  and  feasibility  of  each  idea  is  left  to  the  members  of  the  training  system 
under  analysis  and  the  readers  of  this  thesis.  The  authors  present  their  own  conclusions 
and  recommendations  in  the  next  chapter. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

The  final  chapter  of  this  thesis  ties  together  all  the  information  presented  in  the 
previous  chapters  and  answers  the  investigative  questions.  The  chapter  begins  with  a 
discussion  on  the  issues  surrounding  this  research  and  the  relevance  of  the  research  to  the 
effectiveness  of  the  Aircraft  Maintenance  and  Munitions  Officers  Course  (AMMOC). 

Most  significantly,  this  chapter  summarizes  the  research  findings,  answers  the  investigative 
questions,  and  presents  conclusions  and  recommendations. 

Research  Issues 

The  scope  of  this  thesis  was  quite  broad.  The  study  dealt  with  guidance  and 
support,  which  we  defined  as  policy,  resources,  services,  or  anything  else  that  affects 
AMMOC's  ability  to  accomplish  its  mission.  Because  of  the  broad  scope  of  this  research, 
no  particular  topic  was  scrutinized  in  detail.  Instead,  this  research  attempted  to  provide  a 
comprehensive  view  of  the  training  system  and  to  identify  some  general  areas  for  future 
improvement  efforts.  By  no  means  does  this  thesis  preclude  the  need  for  future  study  in 
the  area  of  technical  training  policy.  To  the  contrary,  presented  in  this  chapter  are  areas  of 
interest  which  should  be  the  subjects  of  intense  scrutiny  by  senior  technical  training 
managers. 

Do  not  expect  to  arrive  at  the  researchers’  conclusions  by  reviewing  only  the 
responses  to  the  second  round  Delphi  questionnaire  presented  in  Appendix  G.  The 
researchers  conclusions  are  based  on  three  primary  sources-the  open-ended  responses  to 
the  first  round  Delphi  questionnaire,  the  close-ended  responses  to  the  second  round,  and 
the  descriptive  model  of  the  training  system  presented  in  Chapter  H.  Several  of  the  topics 
firom  the  first  round  were  so  overwhelming,  we  did  not  re-address  them  in  the  second 
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round;  yet,  the  responses  from  the  first  round  are  significant  and  influenced  our 
conclusions  and  recommendations.  Unfortunately,  we  do  not  present  the  first  round 
responses  in  their  entirety  due  to  the  emodonal  nature  of  some  responses  and  the  need  to 
protect  the  participants’  anonymity.  Excerpts  of  the  first  round  responses  which 
substantiate  findings  presented  in  this  chapter  are  presented  throughout  Chapter  IV. 

Finally,  the  recommendations  provided  in  this  chapter  are  those  of  both  AMMOC 
instructors  and  the  researchers.  We  acknowledge  that  there  are  other  credible  sources  of 
training  expertise;  however,  we  felt  it  was  important  to  provide  the  instructors,  as  the 
"customer  point  of  contact,"  a  fOTum  for  providing  their  concerns  and  ideas  which  will 
hopefully  benefit  AMMOC  and  the  system  that  supports  it. 

Review 

Recall  from  Chapter  I  the  following  research  objective  and  investigative  questions: 

Research  Objective.  Identify  opportunities  to  improve  the  support  and 
guidance  provided  to  the  Aircraft  Maintenance  and  Munitions  Officers  Course 
(AMMOC). 

Investigative  Questions. 

1.  What  areas  related  to  the  guidance  and  support  provided  to  AMMOC 
are  the  strongest  candidates  for  improvement  initiatives? 

2.  How  can  the  identified  areas  be  improved? 

Results.  This  study  was  successful  both  in  identifying  opportunities  to  improve 
the  guidance  and  support  provided  to  AMMOC,  and  in  providing  suggestions  for 
capitalizing  on  these  opportunities.  The  researchers  found  that  opportunities  to  improve 
guidance  and  support  are  rooted  in  "conditions"  that  are  contrary  to  effective  training 
which  exist  under  current  guidance  and  support.  Recommended  improvements  to  current 
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guidance  aiKl  support  are  aimed  at  improving  these  conditions  which  were  cited  by 
AMMOC  instructors. 


Summary  of  Findings 

Feedback  fix)m  the  AMMOC  instructors  provides  strong  evidence  that  the 
eolations  presented  in  Table  5-1  exist  under  the  jMesent  system.  As  will  be  demonstrated 
later  in  this  chapter,  these  conditions  are  also  consistent  with  the  descriptive  model  of  the 
training  system  presented  in  Chapter  II. 

Table  5-1.  Summary  of  Findings 

1.  Communication  between  AMMOC  and  its  customers  is  ineffective. 

2.  Communication  between  AMMOC  and  rest  of  the  training  system  is  ineffective. 

3.  The  training  system  supporting  AMMOC  is  unresponsive. 

4.  InstmctcH^  are  afforded  insufficient  time  to  accomplish  course  development 

5.  Student  scheduling  is  carried  out  without  sufficient  involvement  of  AMMOC. 


The  conditions  in  Table  5-1  translate  into  areas  which  provide  strong  improvement 
opportunities,  which  in  turn,  answer  the  first  investigative  question.  Table  5-2  presents 
these  areas  related  to  the  guidance  and  support  provided  to  AMMOC  which  are  strong 
candidates  for  future  improvement  iiutiatives. 
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Table  5>2.  Candidates  for  Improvement  Initiatives 

1.  Customer  Communications 

2.  Training  System  Internal  Communications 

3.  Training  System  Adaptability 

4.  Course  Development  Policy 

5.  Student  Scheduling  Policy 


improvement  Through  Empowerment 

According  to  an  Air  Force  white  paper.  Air  Force  restructure,  empowerment 
strengthens  lines  of  authority  and  facilitates  accountability  (Department  of  the  Air  Force, 
1991d:  2),  Throughout  the  remainder  of  this  chapter,  we  will  demonstrate  that  to  a  large 
extent,  the  conditions  cited  in  Table  5-1  are  beyond  the  control  of  the  AMMOC  instructor 
staff.  By  empowering  AMMOC  instructors  with  more  authority  and  resources,  these 
conditions  may  be  brought  within  their  control.  If  AMMOC  instructors  are  not 
empowered  to  succeed,  they  cannot  legitimately  be  held  accountable  for  the  course's 
overall  performance. 

Table  5-3  presents  the  recommendations  of  AMMOC  instructors  and  the 
researchers  for  improving  the  conditions  identified  in  Table  5-1.  The  recommendations 
originate  from  the  analysis  of  the  descriptive  model  presented  in  Chapter  II  and  the 
analysis  of  both  the  first  and  second  roimd  Delphi. 

On  the  following  pages,  the  validity  of  each  recommendation  is  substantiated. 
Some  of  the  recommendations  should  be  relatively  easy  to  implement;  however,  others 
may  require  changes  in  organizational  structure,  increases  in  manpower,  and  increases  in 
other  resources.  Such  changes  will  help  empower  AMMOC  to  control  the  people  and 
resources  required  to  develop  and  provide  training  that  is  consistent  with  and  responsive 
to  the  customers'  needs. 
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Table  5-3.  Recommendations  for  Improving  of  AMMCXT’s  Guidance  and  Support 


1.  Determine  and  satisfy  AMMOC's  tme  manpower  requirements,  and  refine  training 
development  and  manning  policy  if  necessary. 

2.  Decentralize  Training  Manager  and  Training  Development  Element  (TDE) 
activities  to  AMMOC. 

3.  Decentralize  Training  Quality  Assessment  activities  to  AMMOC. 

4.  Provide  AMMOC  limited  authority  to  implement  changes  to  CTS  in  the  absence 
ofU&TWs. 

5.  Include  AMMOC  in  HQ  USAF  and  major  command  message  traffic  concerning 
maintenance  policies  and  practices. 

6.  Authorize  and  fund  instructor  TDYs  to  operational  wings. 

7.  Provide  AMMOC  direct  access  to  E-mail. 

8.  Review  AMMOC's  needs  for  student  instructional  materials  and  fund  them 
accordingly. 


Course  Development  Opportunities 

Opportunity  for  course  development  by  the  instructors  is  limited  for  several 
reasons.  To  allow  more  time  for  course  development  activities,  our  recommendations  are 
to  determine  and  satisfy  AMMOC's  true  manpower  requirements,  and  then  to  refine 
training  development  and  manning  policies  if  necessary. 

Round  one  of  the  Delphi  procedure  indicated  that  AMMOC  instructors  view  the 
current  system  of  training  as  unresponsive.  In  round  two  of  the  Delphi,  16  of  17 
instructors  strongly  agreed  that  manning  levels  and  back-to-back  classes  afford  instructors 
inadequate  time  to  conduct  research  and  course  development,  and  rated  this  condition  as  a 
significant  barrier  to  effective  training.  AMMOC  instructors  suggested  that  AMMOC 
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could  become  more  responsive  if  additional  time  were  available  for  instructors  to 
accomplish  course  development. 

The  AMMOC  instructor  staff  presently  consists  of  16  instructors  for  basic 
AMMOC  and  three  short  courses.  The  demands  of  the  short  courses  leave  a  core  of 
approximately  ten  instructco^  who  are  routinely  available  to  operate  and  develop  the  basic 
course.  Recall  from  Chapter  n  that  the  course  design  calls  for  eight-hour  training  days 
and  a  continual  requirement  for  ten  instructors  in  the  classroom.  It  is  common  for  these 
core  instructors  to  spend  10  to  12  weeks  in  the  classroom  before  receiving  a  two-week 
break  during  which  course  development  and  personal  business  may  be  conducted.  This 
simation  significantly  limits  opportunities  to  conduct  course  development  (Dalton,  1994). 

This  problem  is  not  simple,  but  rather  a  product  of  several  interrelated  factors. 
These  factors  include  policies  concerning  training  development  and  manpower,  and  the 
nonavailability  of  volunteers  for  AMMOC  instructor  duty.  A  long-term  solution  to  this 
condition  should  focus  on  the  evaluation  and  refinement  of  such  policies  which  may  be 
inconsistent  with  the  development  and  conduct  of  quality  training.  The  following 
paragraphs  present  a  discussion  of  these  imponant  factors. 

Technical  training  development  policy  is  a  potential  constraint  on  course 
development  time.  According  to  a  draft  of  AETCR  52-1,  Technical  training 
development,  the  standard  training  day  is  based  on  eight  hours  of  training  (Department  of 
the  Air  Force,  1994c:  62).  AMMOC's  course  design  is  consistent  with  this  policy.  If  the 
course  design  accommodated  a  shorter  training  day,  more  time  would  be  available  to 
instructors  for  course  development.  However,  such  a  change  would  require  a  course  with 
fewer  total  hours  of  instruction,  or  one  with  more  training  days  to  provide  the  same 
number  of  hours  of  instruction.  Training  policy  does  permit  up  to  two  hours  per  day  for 
supervised  or  directed  study  which  is  not  currently  included  in  AMMCXU's  course  design 
(Department  of  the  Air  Force,  1993a;  1994c:  62). 
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Another  important  factor  is  the  method  for  determining  AMMOC's  manning 
authorizations.  AMMOC's  manning  authorizations  are  computed  in  accordance  with  a 
iTKxlel  presented  in  ATC  Regulation  25-2,  Technical  training  course  manpower 
standards.  This  nKxlers  parameters  include  course  length,  maximum  class  size,  number  of 
instructors  required  for  each  block  of  instruction,  and  the  annual  trained  personnel 
requirement  (TPR)  (Department  of  the  Air  Force,  1987:  8-11).  An  apparent  underlying 
premise  of  this  model  is  that  there  is  a  great  deal  of  similarity  among  all  technical  training 
courses.  It  is  unclear  to  the  researchers  if  this  manpower  model  considers  the  peculiarity 
of  each  course's  curriculum  and  associated  course  development  requirements.  A  review  of 
Air  Force  Catalog  (AFCAT)  36-2223,  U SAP  formal  schools,  indicates  that  AMMOC's 
curriculum  is  somewhat  broader  than  that  of  most  other  technical  training  courses. 
Assuming  that  AMMOC's  curriculum  is  indeed  broader  than  other  technical  training 
courses,  it  is  logical  to  assume  that  more  research  is  required  to  develop  and  maintain 
AMMOC  curriculum  than  is  required  for  other  courses. 

One  suggestion  offered  during  the  Delphi  was  to  assign  an  additional  cadre  of 
personnel  to  course  development  activities.  This  would  relieve  instructors  of  the  burden 
of  both  instructing  and  conducting  course  development.  Such  a  scheme  is  employed  by 
the  Squadron  Officers  School  (SOS)  at  Maxwell  AFB,  Alabama,  where  approximately  14 
officers  are  assigned  to  course  development  activities  for  a  single  course  which  is  195 
hours  in  length  (Trip,  1994).  In  contrast,  AMMOC  dedicates  no  personnel  to  AMMOC 
course  development  and  maintenance  for  four  courses  with  a  combined  length  of  1 106 
hours  (Dalton,  1994).  Although,  it  is  possible  that  some  differences  exist  between  course 
development  requirements  for  AMMOC  and  SOS,  there  certainly  seems  to  be  a  huge 
inconsistency  between  their  manning  policies. 

Finally,  the  nonavailability  of  volunteers  for  AMMOC  instructor  duty  has  been  a 
major  constraint  in  the  opportunity  for  course  development.  At  the  present  time  AMMOC 
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is  authorized  20  instructors  for  the  basic  and  short  courses.  However,  only  16  instructors 
are  currently  assigned  to  AMMOC  (Dalton,  1994).  Current  policy  on  maintenance  officer 
assignments  is  that  only  volunteers  are  assigned  to  AMMOC  unless  instructor 
authorizations  are  designated  as  "must  fill"  by  AETC.  Once  AETC  has  indicated  the 
positions  are  "must  fill,"  the  remaining  authorizations  are  filled  to  the  current  manning 
level  of  captain  maintenance  officers  throughout  the  rest  of  the  Air  Force,  which  is 
presently  around  95  percent  The  Air  Force  Manpower  and  Personnel  Center  (AFMPC)  is 
currently  in  the  process  of  placing  five  nonvolunteers  into  the  five  vacant  positions  to 
completely  fill  AMMCXI’s  twenty  authorizations  (Adkinson,  1994). 

The  issue  arises  as  to  whether  an  environment  rich  with  nonvolunteers  is  healthy 
for  an  OTganization  interested  in  high  morale  and  quality  training.  This  is  particularly 
important  because  AFMPCs  cunent  practice  is  to  select  officers  with  at  least  fifteen  years 
of  service  for  the  nonvolunteer  assignments  (Adkinson,  1994).  This  level  of  experience 
places  nonvolunteers  among  the  most  senior  of  AMMCXI  insmictors.  As  a  result, 
nonvolunteers  are  more  likely  to  assume  the  role  of  course  supervisor. 

Instead  of  relying  on  nonvolunteers,  perhaps  in  the  long  run,  it  would  be 
advantageous  to  make  AMMOC  a  more  attractive  assignment  for  maintenance  officers. 

We  presented  AMMCXU's  vision  in  Chapter  II,  part  of  which  was  to  be  "the  'place  to  be' 
for  both  instructors  and  students"  (AMMOC,  1992).  How  to  realize  this  vision  is  no 
simple  dilemma.  It  is  possible  that  impressing  students  with  quality  training  would  create 
a  referent  attitude  among  students  towards  AMMOC  that  would  encourage  their  return  as 
instructors.  Furthermore,  if  AMMOC  instructor  duty  were  viewed  as  a  good  career 
opportunity,  a  likely  result  would  be  more  volunteers  for  instructor  duty.  Consider  the 
possibility  of  having  more  volunteers  than  could  be  accepted  by  the  course.  This  would 
facilitate  recruitment  of  the  finest  officers-a  certain  contribution  to  quality  training. 
Unfortunately,  the  cycle  must  be  started  in  some  manner  to  attract  volunteers. 
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Empowering  instmctors  with  additionally  responsibilities  and  the  necessary  resources 
would  be  a  start  In  essence,  satisfy  the  needs  of  the  current  instructors  so  that  they  begin 
nmrketing  the  benefits  of  AMMOC  instructor  duty.  A  complimentary  short-term  solution 
might  be  to  offer  p»-ospective  instructors  some  type  of  incentive  such  as  their  choice  of  a 
follow-on  assignriKnt  Such  incentives  were  suggested  during  the  first  round  of  the 
Delphi  procedure  and  received  strong  support  among  instructors  in  the  second  round. 

Creating  more  time  for  course  development  is  a  complex  issue  which  should  be  the 
subject  of  further  study  that  considers  all  the  relevant  factors  discussed  above.  The  first 
step  in  solving  this  problem  is  determining  AMMOC’s  true  manpower  requirements.  The 
next  step  should  be  the  evaluation  of  technical  training  and  manpower  policies  to 
determine  if  they  are  consistent  with  AMMOC's  requirements.  If  these  policies  are 
inconsistent  with  AMMOC's  needs,  the  policies  should  be  refined.  According  to  Osborne 
and  Gaebler,  authors  of  Reinventing  government,  those  who  serve  the  customer  are  right 
next  to  the  customer  in  terms  of  importance  (1992: 172).  If  the  Air  Force  agrees  with 
Osborne  and  Gaebler's  philosophy,  techrucal  training  policies  should  support  the  needs  of 
those  who  actually  train. 

Decentralization  of  Essential  Training  Support  Activities 

Three  of  the  recommendations  presented  in  Table  5-3  fall  under  the  concept  of 
decentralization  and  empowerment.  They  were  to  decentralize  training  manager  and 
Training  Development  Element  (TDE)  activities  to  AMMOC,  decentralize  Evaluation 
activities  to  AMMOC,  and  provide  AMMOC  limited  authority  to  implement  changes  to 
the  CTS  in  the  absence  of  U&TWs.  Each  is  addressed  in  the  following  sections. 

Training  Manager  and  Training  Development  Element  (TDE)  Activities 
As  explained  in  Chapter  II,  the  training  manager  is  assigned  to  Training  Plans,  a  staff 
function  of  the  362nd  Training  Squadron.  The  training  manager  constitutes  a  major  link  in 
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communications  between  AMMCX),  its  customers,  and  other  components  of  training 
system  such  as  the  Air  Force  Career  Field  Manager  (AFCFM),  Major  Command 
Functional  Managers  (MFMs),  AFMPC,  and  Second  Air  Force  (2AF).  The  training 
manager  is  also  a  focal  point  of  the  training  system  for  critical  activities  affecting 
AMMOC  such  as  student  scheduling  and  Course  Training  Standard  (CTS)  development. 
Although  student  scheduling  and  CTS  development  are  activities  which  significantly  affect 
AMMOC,  AMMOC  lacks  authority  over  them.  This  situation  is  not  consistent  with  the 
Quality  Air  Force  principle  of  "empowerment  at  the  point  of  contact"  introduced  in 
Chapter  I.  By  decentralizing  responsibility  for  these  activities  to  AMMOC,  student 
scheduling,  CTS  development,  and  communications  may  be  improved. 

The  benefits  of  decentralizing  the  training  manager’s  activities  to  AMMOC  are 
significant.  Perhaps  the  greatest  is  the  empowerment  of  AMMOC  over  essential  activities 
and  the  elimination  of  a  potential  constraint  in  the  communication  process.  Remember 
that  AMMOC's  training  manager  is  not  only  responsible  for  AMO  training,  but  also  for 
training  for  the  maintenance  data  systems  analysis  and  the  maintenance  scheduling  career 
fields.  Decentralization  of  the  training  manager's  activities  to  AMMOC  will  help  facilitate 
a  direct  dialog  between  AMMOC,  its  customers,  and  other  components  of  the  training 
system,  a  dialog  which  will  help  assure  that  AMMOC's  interests  are  effectively 
communicated  to  others. 

As  noted  in  Chapter  II,  infrequent  Utilization  and  Training  Workshops  (U&TWs) 
arc  the  forum  for  reviewing  and  updating  the  CTS.  Responsiveness  to  customer  needs 
necessitates  continuous,  effective  communication  between  AMMOC  and  the  AFCFM  and 
MFMs.  However,  training  managers  within  the  82nd  Training  Group  do  not  routinely 
communicate  with  MFMs,  nor  do  they  normally  pursue  changes  to  the  CTS  outside  of  the 
U&TW  process  (Lawlor,  1994).  Because  of  the  infrequency  of  U&TWs  and  the  apparent 
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policy  of  noncommunication  with  MFMs,  communication  between  AMMOC  and  its 
customers  is  limited. 

Given  that  the  training  manager's  activities  are  decentralized  to  AMMCKD,  it  is  also 
logical  to  decentralize  Training  Development  Element  (TDE)  activities  to  AMMOC. 
Recall  from  Chapter  n  that  TDE  is  an  intermediary  between  AMMOC  and  the  training 
manager  in  the  coordination  and  approval  of  some  course  control  documents.  This  liaison 
would  no  longer  be  required.  In  fact,  decentralization  of  TDE  to  AMMOC  would 
eliminate  yet  another  layer  of  the  organization  from  the  communications  process. 

Decentralizing  the  training  manager's  and  TDE  activities  will  help  facilitate  routine 
interaction  between  AMMOC,  AFCFM,  and  MFMs.  Such  interaction  should  contribute 
to  a  more  responsive  training  system.  Figures  5-1  and  5-2  depict  course  development 
communications  before  and  after  decentralization  of  the  training  manager's  activities  to 
AMMOC. 


Figure  5-1.  Model  of  Existing  Communications  System 
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Figure  5-2.  Model  of  Proposed  Communications  System 


Decentralizing  the  training  manager's  activities  also  offers  AMMCXT  the  increased 
influence  over  student  scheduling.  Student  scheduling  is  important  because  it  affects  the 
instructors'  opportunities  to  engage  in  course  development  Instructors  contend  that 
because  of  the  lean  staffing  of  AMMOC  instructors,  and  continuous  back  to  back  classes, 
time  fen-  course  development  is  at  a  premium.  Creative  scheduling  of  incoming  classes 
may  afford  instructors  more  time  outside  of  the  classroom  for  course  development 
activities.  Figures  5-3  and  5-4  below  depict  the  communications  required  for  student 
scheduling  before  and  after  decentralization  of  the  training  manager's  activities  to 
AMMOC. 


Figure  5-3.  Model  of  Current  Scheduling  System  Communications 
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Figure  5-4.  Model  of  Proposed  Scheduling  System  Communications 

Decantralize  Evaluation  Activities  to  AMMQC.  Like  Training  Plans, 
Evaluation  is  a  focal  point  for  essential  activities  that  AMMOC  relies  upon.  As  a  staff 
function  of  the  training  group,  Evaluation  is  the  focal  point  for  the  collection  of  external 
customer  feedback  for  hundreds  of  courses,  including  AMMOC,  which  are  assigned  to  the 
82nd  Training  Group. 

Recall  that  of  the  forms  of  customer  feedback  presented  in  Chapter  II,  none  were 
presenting  AMMOC  with  significant  feedback.  The  most  recent  significant  feedback  to 
AMMOC  was  a  1991  Training  Evaluation  Report  (TER)  of  AMMOC.  Graduate 
Assessment  Surveys  (GASs)  which  are  administered  to  100  percent  of  all  enlisted  training 
graduates  are  not  utilized  at  all  for  AMMOC  graduates.  The  Customer  Support 
Informatirxi  Line  (CSIL),  a  feedback  mechanism  for  customers  in  the  field,  is  unadvertised 
to  AMMOC  graduates  and  is  manned  by  a  single  agency  tasked  with  supporting  literally 
hundreds  of  other  courses  administered  by  the  82nd  Training  Group.  With  the  exception 
of  the  end-of-course  student  critiques,  and  infrequent  U&TWs,  it  appears  that  AMMOC  is 
operating  without  customer  feedback. 
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With  the  assistance  of  TQA,  AMMOC  should  be  empowered  to  design  and 
administer  its  own  program  for  soliciting  continuous  feedback  from  customers. 
Decentralizing  the  collection  of  feedback  to  AMMOC  would  eliminate  an  echelon  of  the 
organization  from  the  feedback  process  and  could  help  strengthen  ties  with  the  customer. 
Because  AMMOC's  program  would  be  concerned  with  only  AMMOC's  customers,  as 
opposed  to  the  customers  of  every  course  in  the  training  group,  AMMOC's  feedback 
program  would  likely  achieve  a  higher  degree  of  customer  focus. 

Provide  Umited  Authority  to  Implement  Changes  to  the  CTS.  The  last 
decentralization  recommendation  is  to  provide  AMMOC  limited  authority  to  implement 
changes  to  the  CTS  in  the  absence  of  U&TWs.  AMMOC  instructors  indicated  their  lack 
of  authority  to  modify  the  CTS  adversely  affects  the  responsiveness  of  the  training  system. 
Because  changes  to  the  CTS  are  not  normally  pursued  outside  of  the  U&TW  foram,  and 
because  U&TWs  are  relatively  infrequent  (seven  years  between  that  last  two),  it  is 
possible  for  AMMOC  to  become  bound  to  a  CTS  which  is  actually  inconsistent  with  real 
training  needs.  If  the  CTS  is  to  accurately  reflect  the  needs  of  the  customers,  it  must  be  a 
"living  document”  which  is  continually  subject  to  review  and  improvement.  AMMOC 
instructors  contend  that  the  curriculum  development  process  could  be  improved  and  the 
training  system  could  become  more  responsive  if  AMMOC  instructors  were  provided 
authority  to  modify  the  CTS  when  changes  in  maintenance  policies  and  practices  in  the 
field  warrant  such  changes. 

The  new  ISD  model  emphasizes  that  ISD  is  a  continuous  process  of  improvement. 
An  important  question  is,  does  limiting  CTS  development  to  U&TWs  contribute  to 
continuous  improvement?  The  most  recent  U&TW  held  in  July  1994  suggests  that  it  does 
not  The  1994  U&TW  was  the  first  in  seven  years.  During  this  workshop,  the 
participants  agreed  to  drastic  revisions  in  course  length  and  curriculum  content  (Broardt, 
1994).  While  this  change  is  welcome,  it  also  provides  evidence  of  the  nonresponsiveness 
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of  this  system  during  the  periods  between  U&TWs.  Instead  of  continuous  incremental 
changes  that  are  encouraged  by  QAF  philosophies,  the  systems  responds  with  large 
intermittent  adjustments.  This  particular  adjustment  is  so  drastic  that  a  full  year  will  elapse 
before  the  new  changes  are  implemented  into  the  course  (Broardt,  1994). 

The  researchers  acknowledge  that  the  CTS  is  a  "contract"  between  customers  and 
AMMOC,  and  do  not  advocate  AMMOC's  management  of  the  CTS  in  the  absence  of 
customer  oversight.  Furthermore,  it  is  evident  from  the  second  round  Delphi  responses 
that  AMMOC  instructors  think  that  the  CTS  should  be  developed  with  the  involvement  of 
maintenance  managers  from  operational  units  and  major  command  staffs.  The  CTS  could 
become  more  responsive  to  customer  needs  by  providing  AMMOC  instructors  authority 
to  modify  the  CTS  with  inputs  of  managers  ftom  the  field  and  the  oversight  of  the 
AFCFM  and  MFMs. 

A  schente  for  achieving  this  would  be  to  allow  AMMOC  to  implement  proposed 
CTS  changes  prior  to  review  and  approval  of  the  AFCFM  and  MFMs.  In  cases  where  the 
proposed  changes  are  disapproved  by  the  AFCFM  and  MFMs,  AMMOC  could 
immediately  reverse  them.  We  suspect  that  the  vast  majority  of  AMMOC's  proposals 
would  be  approved.  This  is  supported  by  the  fact  that  CTS  changes  recommended  by 
AMMOC  instructors  were  accepted  almost  without  exception  during  the  1994  U&TW 
(Broardt,  1994).  Such  a  system  could  be  viable  given  that  training  manager's  activities  are 
decentralized  to  AMMOC  as  recommended  above.  The  likely  result  of  this  system  would 
be  a  far  more  dynamic  curriculum  development  process  that  would  be  more  responsive  to 
the  customers'  training  needs. 

Communications 

From  the  first  round  of  the  £>elphi  procedure,  we  concluded  that  communications 
between  AMMOC,  its  customers,  and  others  involved  in  the  training  system  are 
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ineffective.  Many  instructors  contend  that  this  condition  leads  to  AMMOC's  isolation 
from  the  operational  world,  which  in  turn  leads  to  outdated  instruction.  In  some  cases, 
according  to  AMMOC  instructors,  it  is  smdents  who  inform  the  instructors  of  the  latest 
policies  in  the  field.  Three  recommendations  were  suggested  and  supported  by  AMMOC 
instmctors  for  improving  communications  between  AMMOC  and  the  field.  They  were  to 
have  HQ  USAF  and  major  commands  include  AMMOC  in  message  traffic  concerning 
maintenance  policies  and  practices,  authorize  and  fund  instructor  TDYs  to  operational 
wings,  and  provide  AMMOC  direct  access  to  E-mail.  Each  is  discussed  in  the  following 
sections. 

Include  AMMOC  in  Pertinent  Message  Traffic.  Much  of  AMMOC's 
curriculum  is  related  to  maintenance  policy  and  practices.  Unfortunately,  according  to 
instructors,  AMMOC  is  often  not  included  in  correspondence  involving  impending 
changes  in  policies.  As  a  result,  AMMOC  becomes  alerted  to  such  changes  through 
informal  channels,  or  after  changes  have  been  incorporated  into  practice.  This  simation 
places  AMMOC  in  a  reactionary  mode.  According  to  one  AMMOC  instructor,  "the 
credibility  of  instructors  is  challenged  when  we  brief  policies  that  are  no  longer  in  effect  or 
talk  about  how  great  changes  are  working  when  in  reality  it  is  garbage."  By  including 
AMMOC  as  an  informational  addressee,  AMMOC  could  become  aware  of  impending 
policy  changes  and  incorporate  them  into  lesson  plans  prior  to  their  taking  effect.  This 
would  not  only  improve  the  responsiveness  of  training  system,  but  would  contribute  to 
AMMOC’s  vision  of  "be[ing]  on  the  leading  edge  and  [being]  recognized  as  the  authority 
in  the  aircraft  maintenance  and  munitions  management  field"  (AMMOC,  1992). 

Authorize  and  Fund  Instructor  TDYs.  Instructors  believe  that  TDYs  to 
operational  wings  would  help  keep  instructors  abreast  of  current  policies  and  practices  in 
the  field.  Furthermore,  such  visits  to  the  field  should  contribute  to  a  stronger  rapport 
between  AMMOC  and  its  customers. 
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Provide  Direct  Access  to  E-mail.  E-mail  can  provide  AMMOC  direct  contact 


with  its  customers  and  others  involved  in  AMO  training  throughout  the  world.  E-mail 
offers  advantages  over  normal  telephone  service.  First,  it  is  unnecessary  for  both  parties 
to  be  available  simultaneously  to  communicate.  This  feature  is  important  because  most 
AMMOC  instructors  are  unavailable  except  at  lunch,  before  the  beginning  of  the  duty  day, 
and  after  the  end  of  duty  day.  These  are  times  when  those  with  whom  instructors  wish  to 
communicate  are  often  unavailable. 

Consider  the  opportunities  available  to  AMMOC  if  they  had  E-mail.  AMMOC 
could  create  a  compilation  of  E-mail  addresses  of  those  maintenance  officers  who  have  E- 
mail  access  and  are  interested  in  influencing  the  direction  AMMOC  is  taking.  E-mail 
could  serve  as  a  continuous  source  of  customer  feedback.  When  AMMOC  has  an  issue  to 
resolve,  it  could  present  the  issue  to  AMOs  in  the  field,  and  get  practically  instantaneous 
feedback  from  a  cross-section  of  officers  around  the  world.  E-mail  would  serve  as  a  type 
of  Customer  Service  Information  Line  (CSIL)  that  is  directly  accessible  to  AMMOC  as 
opposed  to  the  current  CSIL  which  is  centrally  managed  and  unpublicized  to  AMOs. 

A  second  advantage  over  normal  telephone  service  is  that  E-mail  would  provide  a 
medium  through  which  potential  insuuctional  materials  may  be  quickly  and  conveniently 
transmitted.  The  implications  on  course  administration  could  be  significant.  If  the 
recommendation  presented  in  this  thesis  to  provide  limited  authority  to  AMMOC  to 
implement  changes  to  the  CTS  were  accepted.  E-mail  could  provide  the  means  for  the 
final  approval  of  such  changes.  With  all  the  appro{niate  players  on  line,  the  U&TW 
process  could  be  a  continuous  process.  The  merits  of  E-mail  are  overwhelming. 

Student  Instructional  Materials 

The  last  specific  improvement  recommendation  is  to  review  AMMOC's  needs  for 
student  instmctional  materials  and  fund  the  requirements  accordingly.  One  hundred 
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percent  of  instructors  agreed  that  funding  is  inadequate  for  student  instructional  materials 
such  as  study  guides  and  student  handouts.  We  assume  that  the  resources  dedicated  to 
developing  and  administering  training  are  intended  to  culminate  in  the  actual  training  of 
students.  Carl  Albrecht,  author  of  Service  within,  would  likely  describe  the  actual  delivery 
of  training  as  a  "moment  of  truth."  "Moments  of  truth"  are  the  critical  times  when  those 
who  serve  the  customer  either  succeed  or  faiL  All  the  time  and  effcai  dedicated  to  training 
development  is  essentially  wasted  if  at  the  "moment  of  truth"  you  are  unprepared  to  serve. 
AMMOCs  needs  for  instructional  material  should  be  reviewed,  and  legitimate 
requirements  for  instructional  materials  should  be  funded. 

Quality  Air  Force  Revisited 

Recall  from  Chapter  I  the  Air  Force's  emphasis  on  Quality  Air  Force  (QAF).  Not 
only  is  QAF  important  in  AMMOC's  effectiveness,  but  AMMOC  plays  an  important  role 
in  the  proliferation  of  QAF  principles  throughout  the  Air  Force.  If  QAF  is  a  philosophy 
that  is  to  permeate  all  levels  of  the  Air  Force,  it  must  be  instilled  in  our  officers  from  the 
beginnings  of  their  careers,  during  training.  QAF  values  instilled  in  our  AMOs  are  carried 
forth  to  aircraft  maintenance  organizations  in  the  field  where  the  QAF  philosophies  may  be 
further  spread  to  others  throughout  the  Air  Force.  However,  it  is  unlikely  that  QAF 
principles  can  be  instilled  during  training  if  the  organization  which  trains  the  officers  does 
not  exemplify  QAF. 

QAF  emphasizes  "customer  focus"  and  "empowerment  at  the  point  of  contact."  It 
is  evident  from  the  AMMOC  instructors  feedback  that  the  present  support  and  guidance 
provided  to  AMMOC  is  deficient  in  both  areas.  To  achieve  "customer  focus"  and 
"empowerment  at  the  point  of  contact"  requires  decentralization  of  authority,  and 
empowerment  of  AMMOC. 
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It  is  evident  from  the  successes  of  aircraft  maintenance  organizations  in  the  field 
that  empowerment  of  AMOs  has  positive  results.  Virtually  all  aircraft  maintenance 
organizations  today  are  based  on  decentralized  concepts  that  empower  aircraft 
maintenance  officers  with  the  authority  and  resources  to  accomplish  their  missions.  It  is 
ironic  that  AMMOC  instructors  who  are  charged  with  training  AMOs  to  lead  and  manage 
decentralized  organizations  are  in  fact  members  of  centralized  organizations. 

Areas  for  Further  Research 

Manpower.  We  indicated  earlier  in  this  chapter  that  the  manpower  model  and 
the  appropriateness  of  the  model  for  AMMOC  should  be  evaluated.  We  hypothesize  that 
the  model  does  not  adequately  account  for  the  breath  of  the  AMMOC  curriculum  and  the 
ensuing  course  development  requirements  in  establishing  the  manpower  authorizations.  In 
any  case,  if  the  training  manager  and  Evaluadon  fiinctions  are  decentralized  and  AMMOC 
is  given  the  authority  to  manage  a  portion  of  the  CTS  development  process  as 
recommended,  the  implications  associated  with  the  manpower  requirements  need  to  be 
evaluated. 

Facilities.  In  the  general  issues  section  of  the  Delphi's  second  round,  AMMOC 
instructors  unanimously  indicated  that  facilities  should  be  improved.  We  did  not 
specifically  pursue  the  causes  for  this  position.  Commentary  from  the  first  and  second 
round  indicated  that  part  of  the  problem  was  in  the  inadequacies  of  the  heating  and 
cooling  system  which  leads  to  an  ineffective  learning  environment  One  of  our  first  round 
validation  participants  also  indicated  that  some  training  effectiyeness  and  efficiency 
improvements  could  be  obtained  through  a  facilities  upgrade-for  example,  an 
auditorium/lecture  hall  where  mass  briefings  could  be  held  when  guest  speakers  are 
available.  We  believe  it  would  be  beneficial  to  pursue  an  improvement  analysis  of 
AMMOC's  facilities. 
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AMMOC  Evaluation.  We  indicated  in  Chapter  IV,  that  AMMOC  instructors 
seemed  unaware  of  a  systematic  method  of  evaluating  AMMOC ’s  performance.  Our 
methodology  did  not  create  a  consensus  on  how  these  measurements  might  be  improved 
nor  what  measurements  should  be  used.  Future  research  should  focus  on  determining  the 
^)propriate  measures  for  evaluating  AMMOC’s  effectiveness  and  efficiency. 

Apolication  of  this  Research  to  Other  Areas  of  Technical  Training 
Though  we  applied  this  research  to  AMMOC,  we  believe  our  methodology,  findings,  and 
recommendations  can  be  applied  to  other  technical  training  courses,  most  especially  to 
other  officer  technical  training  courses.  We  also  believe  aspects  of  our  methodology  can 
be  applied  to  AMMOC  and  other  courses  on  an  ongoing  basis  as  a  means  for  soliciting 
feedback  from  the  training  system's  internal  customers,  the  instructors.  We  are  unaware 
of  any  existing  formal  means  for  soliciting  feedback  from  the  these  valuable  system 
resources. 

Chapter  Summary 

The  objective  of  this  research  was  to  identify  opportunities  to  improve  the  support 
and  guidance  provided  to  the  Aircraft  Maintenance  and  Munitions  Officers  Course.  The 
intent  of  this  objective  was  to  ultimately  improve  the  product  of  the  course  by  focusing  on 
the  internal  training  system  processes  supporting  AMMOC.  Five  areas  related  to  the 
guidance  and  support  provided  to  AMMOC  were  identified  as  strong  candidates  for 
improvement  initiatives.  They  were: 

1.  Customer  Communications 

2.  Training  System  Internal  Communications 

3.  Training  System  Adaptability 

4.  Course  Envelopment  Policy 

5.  Student  Scheduling  Policy 
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Eight  recommendations  intended  to  improve  the  areas  identified  above  were 
presented  and  discussed.  The  specific  recomn^ndadons  were  to: 

1.  Determine  and  satisfy  AMMOC's  true  manpower  requirements,  and 
refine  training  development  and  manning  policy  if  necessary. 

2.  Decentralize  Training  Manager  and  Training  Development  Element 
(TDE)  activities  to  AMMOC. 

3.  Decentrahze  Training  Quality  Assessment  activities  to  AMMOC. 

4.  Provide  AMMOC  limited  authority  to  implement  changes  to  CTS  in  the 
absence  of  U&TWs. 

5.  Include  AMMOC  in  HQ  USAF  and  major  command  message  traffic 
concerning  maintenance  policies  and  practices. 

6.  Authorize  and  fund  instructor  TDYs  to  operational  wings. 

7.  Provide  AMMOC  direct  access  to  E-mail. 

8.  Review  AMMOC's  needs  for  student  instructional  materials  and  fund 
them  accordingly. 

It  is  our  hope  that  this  thesis  has  provided  valuable  information  and 
recommendations  for  the  improvement  of  the  Aircraft  Maintenance  and  Munitions  Officers 
Course.  Unfortunately,  we,  the  researchers  cannot  control  the  implementation  of  the 
recomrrtendations  provided.  This  is  left  to  the  training  professionals.  We  hope  we  have 
provided  sufficient  justification  and  information  to  serve  as  a  catalyst  for  change.  After 
all,  the  world  is  changing  and  forcing  us  to  train  our  maintenance  officers  as  effectively 
and  efficiently  as  possible  so  that  they  will  be  prepared  to  meet  the  challenges  of  the  future 
as  we  pursue  our  vision  of  "Global  Reach  -  Global  Power." 
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Appendix  A:  Glossary  of  Acronyms 


2AF 

Second  Air  Force 

AETC 

Air  Educadcm  and  Training  Command 

AETCR 

AETC  Regulation 

AFB 

Air  Force  Base 

AFCAT 

Air  Force  Catalog 

AFCFM 

Air  Force  Career  Field  Manager 

AFTT 

Air  Force  Institute  of  Technology 

AFMPC 

Air  Force  Manpower  and  Persoimel  Center 

APR 

Air  Force  Regulation 

AFRES 

Air  Force  Reserve 

AFS 

Air  Force  Specialty 

AFSC 

Air  Force  Specialty  Code 

AFTMS 

Air  Force  Training  Management  System 

AMMO 

Aiicrafit  Maintenance  and  Munitions  Officer 

AMMOC 

Aircraft  Maintenance  and  Munitions  Officers  Course 

AMO 

Aircraft  Maintenance  Officer 

AMOC 

Aircraft  Maintenance  Officers  0>urse 

AMU 

Aircraft  Maintenance  Unit 

ANG 

Air  National  Guard 

ATC 

Air  Training  Command 

BMW 

Bombardment  Wing 

CAMS 

Core  Automated  Maintenance  System 

CC 

Commander  or  Course  Chart 

CDC 

Career  Development  Course 

CFETP 

Career  Field  Education  and  Training  Plan 

CSIL 

Customer  Service  Information  Line 

CTS 

Course  Training  Standard 

A-l 


DCM 

DOD 

DSN 


Deputy  Commander  for  Maintenance 
Department  of  Defense 
Defense  Switchboard  Network 


FEQ 

GAS 

HQUSAF 

ISD 

LGs 

MAJCOM 

MFM 

MOC 

MPC 

MX 

QIC 

OSR 

POC 

POI 

QAF 

SATAF 

SMART 

SME 

SOS 

SQ 

TOB 

TDE 

TOY 


Field  Evaluation  Questionnaire 
Graduate  Assessment  Survey 
Headquarters  United  States  Air  Force 
Instmctional  Systems  Development 
Lx}gistics  Commanders 
Major  Command 

Major  Command  Functional  Manager 
Maintenance  Operations  Center 
Air  Force  Manpower  and  Personnel  Center 
Maintenance 

Officer  in  Charge 
Occupational  Survey  Report 

Point  of  Contact 
Plan  of  Instruction 

Quality  Air  Force 

Site  Activation  Task  Force 

Simulated  Modeling  fOT  Allocatitm  of  Resources  for  Training 
Subject  Matter  Expert 
Squadron  Officers  School 
Squadron 

Training  Development  Branch  (Element) 

Training  Development  Elemoit  (Branch) 

Temporary  Duty 
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TER 

TG 

TPR 

TQA 

TQAP 

TQM 

TQR 

U&TW 

USAF 

WPAFB 


Training  Evaluation  Report 
Training  Group 

Trained  Personnel  Requirement 
Training  Quality  Assessment 
Training  Quality  Assessment  Program 
Total  Quality  Management 
Training  Quality  Report 

Utilization  and  Training  Workshop 
United  States  Air  Force 

Wright-Patterson  Air  Force  Base 
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11  APR  1994 


MEMORANDUM  FOR  82nd  Training  Group  Commander 

620  9th  Avenue  Suite  1 
Sheppard  AFB,  TX  76311-2334 
ATTENTION;  Colonel  Loren  Reno 

FROM:  HQUSAF/LGMM 

1030  Air  Force  Pentagon 
Washington,  DC  20330-1030 

SUBJECT;  Request  for  Research  Support  -  ACTION  MEMORANDUM 


1.  Captains  Jeffrey  T.  Acred  and  James  R.  Witter,  both  graduate  students  at  the 
Air  Force  Institute  of  Technology,  are  conducting  research  on  Aircraft 
Maintenance  Officer  (AMO)  training.  The  focus  of  their  study  is  the  training 
support  infrastructure  for  the  Aircraft  Maintenance  and  Munitions  Officers  Course 
(AMMOC).  An  imponant  part  of  their  research  involves  obtaining  the  opinions  of 
AMMCK;  instructors  on  how  present  Air  Force  and  Air  Education  and  Training 
Command  (AETC)  guidance  and  policy  affect  their  ability  to  provide  effective 
training.  We  are  requesting  your  cooperation  and  support  so  that  Captains  Acred 
and  Witter  may  work  directly  with  the  AMMOC  instructor  staff. 

2.  Captains  Acred  and  Winer's  research  will  have  a  negligible  impact  on  your  daily 
training  operations.  Most  of  their  information  can  be  obtained  by  telephone  and 
through  the  mail.  The  data  for  this  study  will  be  generated  by  the  Delphi  method 
(atch  1)  involving  two  iterations  and,  if  necessary,  a  few  interviews  with  selected 
members  of  your  staff. 

3.  This  research  provides  a  unique  opportunity  for  AMMOC  personnel  to  voice 
their  opinions  on  what  resources,  guidance,  and  authority  should  be  given  to 
AMMOC  to  produce  well-trained  aircraft  maintenance  officers.  Their  expertise 
may  well  contribute  to  future  policy  regarding  AMO  training,  and  possibly  to 
policy  regarding  technical  training  in  other  disciplines.  If  desired,  a  summary  of 
Captain  Acred's  and  Captain  Witter's  research  will  be  provided  to  you  when  it  is 
completed  in  the  Fall  of  1994. 


B-1 


4.  Your  suppon  of  this  research  will  be  greatly  appreciated.  If  you  have  any 
questions  concerning  this  request,  please  contact  either  Captain  Acred  or  Captain 
Witter  at  DSN  785-7777,  extension  2405,  or  their  thesis  advisors.  Dr.  Guy  Shane 
(extension  3347)  or  Lieutenant  Colonel  Rodney  Rice  (extension  3345). 


5.  Please  indicate  your  approval  or  disapproval  of  this  request  by  indorsing  below 


MARK  B.  RODDY,  Colonel,  USAF 
Chief,  Maintenance  Policy  Division 


Attachments: 

1.  Delphi  Technique 

2.  Return  Envelope 

25  APR  1994 

1st  Ind,  82  TG/CC 

TO:  AFTT/LAA  (Capt  Acred/Witter) 

2950  P  Street 

Wright-Patterson  AFB,  OH  45433-7765 
Approve/Diaapprove 


LOREN  M.  RENO,  Colonel,  USAF 
Commander,  82d  Training  Group 


(This  letter  was  electronically  recreated  and  formatted  to  comply  with  the  AFIT 
Style  Guide  for  Theses  and  Dissertations) 
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Appqpdfeg;  .PrgqugstionMiw  NQtfflcatlQn  Lgttgr 


2  May  94 

MEMORANDUM  FOR  362TS/RMA 

527  G  Avenue 

Sheppard  AFB,  TX  7631 1-2835 
ATTENTION:  CaptDoe 

FROM:  AFTT/LAA 
2950  P  Street 

Wright-Patterson  AFB  OH  45433-7765 

SUBJECT:  Request  for  Research  Support  -  INFORMATION  MEMORANDUM 


1.  As  an  expert  in  Aircraft  Maintenance  Officer  (AMO)  training,  your  help  is  needed  in 
completing  important  research  which  is  sponsored  by  Air  Staff  and  supported  by  your 
group  commander,  Colonel  Reno.  The  subject  of  our  research  is  AMMOC  support.  We 
define  support  as  the  resources,  guidance,  and  authority  needed  by  AMMOC  to 
accomplish  its  mission.  It  is  our  position  that  those  who  are  tasked  with  the  training  of 
AMOs  are  the  most  qualified  to  identify  opportunities  to  improve  AMO  training  support. 

2.  In  a  few  days,  you  will  receive  a  Delphi  questionnaire.  Delphi  is  a  technique  for 
soliciting  opinions  and  building  consensus  among  experts.  We  will  seek  your  expertise  on 
a  number  of  issues  regarding  AMMOC  support  that  have  been  raised  as  areas  for 
improvement  by  AMMOC  instructors  within  the  past  two  years.  Ultimately,  our  goal  is  to 
solicit  and  present  your  ideas  pertaining  to  AMO  training  to  the  Air  Force  in  such  a 
manner  that  it  will  benefit  you. 

3.  We  will  present  this  thesis  research  to  the  Air  Staff  and  Air  Force  Institute  of 
Technology  in  partial  fulfilment  of  the  requirements  for  our  master's  degrees.  The  success 
of  our  research  hinges  on  the  accurate  representation  of  your  ideas.  Therefore,  your 
participation  is  vital.  We  greatly  appreciate  your  cooperation. 


SIGNED 

JEFFREY  T.  ACRED,  Capt,  US AF 
Graduate  Student,  Air  Force 
Institute  of  Technology 


SIGNED 

JAMES  R.  WITTER,  Capt,  USAF 
Graduate  Student,  Air  Force 
Institute  of  Technology 
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6  May  1994 

MEMORANDUM  FOR  362TS/RMA 

527  G  Avenue 

Sheppard  AFB.TX  76311-2835 
ATTENTION:  CaptDoe 

FROM:  HQUSAF/LGMM 

1030  Air  Force  Pentagon 
Washington,  DC  20330-1030 

SUBJECT:  Training  Support  Questionnaire  -  ACTION  MEMORANDUM 


1.  Headquarters  USAF,  Maintenance  Policy  Division,  with  the  support  of  your  group 
commander,  is  sponsoring  research  on  the  training  support  that  is  provided  to  AMMOC. 
Attached  is  a  questionnaire  to  obtain  your  opinion  as  an  aircraft  maintenance  officer 
training  expert  on  how  this  training  support  can  be  improved.  The  questionnaire  should 
require  no  more  than  a  couple  of  hours.  We  encourage  you  to  complete  this  questionnaire 
in  a  few  short  sessions  instead  of  all  at  once. 

2.  This  is  the  first  of  two  questionnaires.  The  responses  of  all  participants  from  this 
questionnaire  will  be  combined  in  the  second  questionnaire  for  your  assessment.  You  will 
receive  the  second  questionnaire  in  approximately  six  weeks. 

3.  We  realize  that  your  participation  in  this  research  is  purely  voluntary  and  we  appreciate 
your  support.  Because  this  research  will  be  presented  to  Air  Staff,  it  is  possible  that  your 
qnnions  will  have  an  impact  on  important  decisions  regarding  future  policy  and  guidance 
provided  to  AMMOC  and  to  other  providers  of  technical  training.  Please  complete  and 
return  dus  questionnaire  within  two  weeks. 

4.  If  you  have  any  questions  regarding  this  questionnaire  or  this  research,  please  call 
either  Captain  Acred  or  Captain  Witter  at  DSN  785-7777,  extension  2405. 


SIGNED 

MARK  B.  RODDY,  Colonel,  USAF 
Chief,  Maintenance  Policy  Division 
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Attachments: 

1.  Training  Support  Questionnaire 

2.  Return  Envelope 


(Original  letter  fit  on  one  page.  Format  was  changed  to  comply  with  the  AFIT  Style 
Guide  for  Theses  and  Dissertations) 
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AIRCRAFT  MAINTENANCE  OFFICER 
TRAINING  SUPPORT 
QUESTIONNAIRE 


FIRST-ROUND  RESPONSE  REQUEST 


NOTE:  ALL  RESPONSES  IN  THIS  QUESTIONNAIRE  WILL  REMAIN 

ANONYMOUS 


PLEASE  R3BTURN  THIS  QUESTIONNAIRB  IN  THE  PROVIDED  ENVELOPE  BY; 

nMAYvm 
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The  purpose  of  this  questionnaire  is  to  acquire  your  expert  opinion  concerning  the  support 
provided  to  the  Aircraft  Maintenance  and  Munitions  Officers  Course  (AMMOC).  This  is 
an  opportunity  to  express  your  opinions  anonymously  and  to  provide  valuable  feedback  to 
HQ  USAF,  Air  Education  and  Training  Command,  and  your  group.  The  topics  addressed 
in  this  questionnaire  were  identified  by  AMMOC  personnel  during  a  Quality  Improvement 
workshop  conducted  at  Chanute  AFB  in  Septemb^  1992.  While  your  participation  is 
purely  voluntary,  please  realize  that  your  input  is  vital  to  the  successfiil  completion  of  this 
worthwhile  research. 

Section  A:  Background 

Please  check  the  blocks  in  the  following  questions  which  apply  to  you. 

1.  civilian  _  military _ 

2.  Years  of  instructor  experience  (Cumulative) 

_ none 

_ less  than  2 

_ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_  6  or  more 

3.  Years  as  an  AMMOC  Instructor 

_  none 

_  less  than  2 

_ nK)re  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_ 6  or  more 

4.  Excluding  instructor  time,  years  of  experience  as  an  aircraft  maintenance  officer 

_  none 

_  less  than  2 

_ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_  6  or  more 
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5.  ExcliKling  instructor  dnK,  years  of  experience  as  an  aircraft  maintenance  technician 


_ none 

_ less  than  2 

_ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_ 6  or  more 

6.  Years  of  organizational^teimediate  level  maintenance  experience  (Include  both 
managerial  and  technical  experience.  Do  not  include  instructional  tours) 

_ none 

_ less  than  2 

_ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_ 6  or  more 

7.  Years  of  depot  level  maintenance  experience 

_ none 

_ less  than  2 

_ _ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_ 6  or  more 

8.  Years  since  you  filled  one  of  the  positions  in  6  or  7  above 

_ not  applicable  (have  not  filled  a  position  identified  in  questions  6  or  7) 

_ less  than  2 

_ more  than  2,  but  less  than  4 

_ more  than  4,  but  less  than  6 

_ 6  or  more 

9.  TheC^uality  Air  Force  philosophy  advocates  six  basic  principles:  Leadership 
Involvement,  Dedication  to  Mission,  Respea  of  the  Individual,  Decentralized 
Organization,  Empowerment  at  the  Pbint  of  Contact,  and  Management  by  Fact  Were  you 
aware  of  these  principles? 

_ Yes  _  No 

10.  In  an  effort  to  become  more  efficient  and  effiM;tive,  DOD  is  encouraging,  and  in  some 
cases  directing,  improved  business  practices.  Were  you  aware  of  these  initiatives? 

_ Yes  _ No 
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11.  In  1992,  a  quality  improvement  workshop  was  conducted  at  Chanute  AFB  during 
which  AMMOC  personnel  formulated  mission  and  vision  statements.  Also  identified  were 
potential  areas  for  improvement  that  were  beyond  the  control  of  AMMOC.  Were  you  a 
participant  in  this  workshop? 

_  Yes  _ No 
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Section  B:  General  Quality  Issues 
1.  CUSTOMER  ORIENTATION. 

a.  Who  is  AMMOC's  primary  customer? 


b.  How  does  AMMOC  communicate  with  its  primary  customer? 


2.  COMMUNICATIONS. 

a.  In  your  opinion,  how  effective  are  die  lines  of  communication  between 
AMMOC  and  the  various  evganizations  (Operational  MAJCOMs,  MFC,  AETC,  Air  Staff, 
etc.)  involved  in  the  system  of  training  and  training  development? 


b.  How  can  communications  be  improved? 
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3.  ADAPTABILITY  TO  CHANGE 

a.  How  responsive  is  the  present  system  of  training  and  training  development  to 
incorporating  changes  in  maintenance  and  management  concepts  employed  in  the  field? 


b.  How  can  this  system  become  more  responsive? 


c.  How  flexible  is  the  present  system  of  training  and  training  development  to 
incorporating  changes  in  maintenance  and  management  concepts  employed  in  the  field? 


d.  How  can  this  system  become  more  flexible? 
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4.  INSTRUCTOR  PERFORMANCE  EVALUATION 


a.  How  is  the  performance  of  individual  instructors  measured  and  evaluated? 


b.  What  are  the  strengths  and  weaknesses  of  these  measiues? 


c.  How  can  AMMOC  instructor  performance  evaluations  be  improved? 
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5.  AMMCX:  PERFORMANCE  EVALUATION. 


a.  How  is  the  performance  of  the  AMMO  Course  measured  and  evaluated? 


b.  What  are  the  strengths  and  weaknesses  of  these  measures? 


c.  How  should  the  AMMO  Course  be  evaluated? 
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6.  EMPOWERMENT 


a.  As  an  instructor,  what  authority  should  you  have  which  you  currently  lack? 
Why  should  you  have  this  authority  (what  advantage  is  gained)? 


b.  What  restrictions  or  requirements  have  been  placed  on  AMMOC  as  a  whole,  by 
regulations  or  any  other  guidance,  that  make  training  more  difficult? 


c.  (Tan  the  constraints  in  6b.  above  be  removed  without  adversely  affecting  the 
quality  of  training? 
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Section  C:  Specific  Quality  Issues 
1.  CRITIQUE  PROGRAM. 


a.  What  are  the  strengths  and  weaknesses  of  the  student  critique  program? 


b.  If  you  could  improve  the  usefulness  of  the  student  critique  program,  what 
would  you  change,  and  how? 
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2.  COURSE  OBJECTIVE  DEVELOPMENT. 


a.  What  are  the  strengths  and  weaknesses  of  the  current  method  of  developing 
course  objectives? 


b.  Do  you  know  of  a  better  method  for  developing  course  objectives? 
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3.  COURSE  CONTROL  DOCUMENTS  (Course  Training  Standards,  Course  Training 
Charts,  and  Plans  of  Instruction). 


a.  What  are  the  strengths  and  weaknesses  of  course  control  documents? 


b.  If  you  could  improve  the  usefulness  of  the  course  control  documents  by 
changing  current  regulations  or  guidance,  what  would  you  change,  and  how? 
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4.  CLASS  HOURS,  TIME  FOR  RESEARCH  AND  COURSE  ADMINISTRATION. 

a.  How  do  policies  governing  the  minimum  length  of  the  training  day  affect  the 
amount  of  time  you  have  for  research  and  course  administration? 


b.  What  do  you  diink  should  be  the  policy  on  how  instructors  manage  class  time? 

Why? 


c.  What  should  be  the  minimum  number  of  classroom  hours  per  day?  Should 
there  be  a  standard  training  day? 
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S.  SCHEDULING.  What  role,  if  any,  should  AMMOC  play  in  die  determination  of  class 
sizes,  Stan  dates,  and  graduation  dates? 


6.  ADMINISTRATION.  Are  there  administrative  tasks  imposed  on  AMMOC  that  can 
be  streamlined  or  eliminated  without  adversely  affecting  training? 
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Section  D:  General  Conunoits 


1.  What  other  ideas  do  you  have  for  improving  the  support  provided  to  AMMOC? 


2.  Any  other  comments. 
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20  June  94 


MEMORANDUM  FOR  362TS/RMA 

527  G  Avenue 

Sheppaid  AFB,  TX  76311-2835 
ATTENTION:  CaptDoe 

FROM;  AFTT/LAA 
2950  P  Street 

Wright-Patterson  AFB  OH  45433-7765 
SUBJECT:  Second  Round  Training  Support  Questionnaire 


1.  About  one  month  ago,  with  the  support  of  Air  Staff  and  your  group  commander,  all 
AMMOC  instmctors  were  asked  to  participate  in  a  Delphi  questionnaire  procedure  for 
improving  the  policy  and  guidance  provided  to  AMMOC.  Seventeen  of  nineteen 
instructors  returned  their  questionnaires.  We  are  very  appreciative  of  your  participation. 

2.  We  are  requesting  that  you  complete  the  attached  second  round  questionnaire.  The 
purpose  of  the  second  round  is  to  determine  your  level  of  agreement  with  the  Erst  round 
responses  from  the  other  AMMOC  instructors.  This  round  is  multiple  choice,  and  should 
require  only  about  an  hour  to  complete. 

3.  You  will  notice  that  a  few  sections  from  the  first  round  are  not  included  in  this 
questionnaire.  The  answers  on  those  questions  were  so  strong  we  did  not  feel  any 
additional  clarification  would  be  necessary.  Areas  of  agreement  included  the  fact  that 
cmnmunications  with  external  agencies  may  be  improved  significantly  and  that  AMMOC 
should  play  a  greater  role  in  scheduling  classes.  To  shorten  this  second  round 
questionnaire,  we  combined  similar  fint-round  responses.  Although  your  exact  words 
may  not  be  present,  your  basic  ideas  are  represented.. 

4.  We  realize  that  your  participation  in  this  research  is  purely  voluntary  and  we  appreciate 
your  support.  Because  this  research  will  be  presented  to  Air  Staff,  it  is  possible  that  your 
(pinions  will  have  an  impact  on  important  decisions  regarding  future  policy  and  guidance 
provided  to  AMMOC  and  to  other  providers  of  technical  training. 
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5.  In  advance,  thank-you  for  participating  in  this  second  round.  If  you  have  any  questions 
regarding  this  questionnaire  or  this  research,  please  call  either  of  us  at  DSN  785-7777, 
extension  2405. 


SIGNED 

JEFFREY  T.  ACRED,  Capt,  USAF 
Graduate  Student,  Air  Force 
Institute  of  Technology 


SIGNED 

JAMES  R.  WITTER,  Capt,  USAF 
Graduate  Student,  Air  Force 
Institute  of  Technology 


Attachments: 

1.  Second  Round  Training  Support  Questionnaire 

2.  Return  Envelope 


(Original  letter  fit  on  one  page.  Format  was  changed  u>  comply  with  the  AFTT  Styie 
Guide  for  Theses  and  Dissertations) 
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AIRCRAFT  MAINTENANCE  OFFICER 
TRAINING  SUPPORT 
QUESTIONNAIRE 


SECOND-ROUND  RESPONSE  REQUEST 


NOTE:  ALL  RESPONSES  IN  TfflS  QUESTIONNAIRE  WILL  REMAIN 

ANONYMOUS 
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Aircraft  Maintenance  Officer  (AMO)  Training  Support  Questionnaire 

Second  Round 

The  purpose  of  this  second  questionnaire  is  to  determine  your  level  of  agreement  with  the 
comments  submitted  by  other  respondents  to  the  first  round  questionnaire  on  the  support 
provided  to  the  Aircraft  Maintenance  and  Munitions  Officers  Course  (AMMOC).  Again, 
this  is  an  opportunity  to  express  your  opinions  anonymously  and  to  provide  valuable 
feedback  to  HQ  USAF,  Air  Education  and  Training  Command,  and  your  group.  This  is 
the  last  round.  If  you  have  any  additional  thoughts,  please  feel  ftee  to  add  comments 
anywhere  on  the  questionnaire.  Reference  a  particular  response  if  appropriate.  While 
your  participation  is  purely  voluntary,  please  realize  that  your  input  is  vital  to  the 
successful  completion  of  this  worthwhile  research. 

Section  A:  Background 

Though  you  provided  background  information  in  the  first  round,  please  take  a  moment  to 
check  the  blocks  in  the  following  questions  which  apply  to  you. 

1.  civilian  6  military  11 

2.  Years  of  instructor  experience  (Cumulative) 

_  none 

6  less  than  2 

5  more  than  2,  but  less  than  4 
_ more  than  4,  but  less  than  6 

6  6  or  more 

3.  Years  as  an  AMMOC  Instructor 

_  none 

9  less  than  2 

5  mcMre  than  2,  but  less  than  4 
_ mc»e  than  4,  but  less  than  6 

3  6  or  more 

4.  Excluding  instructor  time,  years  of  experience  as  an  aircraft  maintenance  officer 

6  none 
_ less  than  2 

2  mwe  than  2,  but  less  than  4 

2  more  than  4,  but  less  than  6 

7  6  or  more 
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5.  Excluding  instructor  time,  years  of  experience  as  an  aircraft  maintenance  technician 


9  none 

1  less  than  2 

_ more  than  2,  but  less  than  4 

2  more  than  4,  but  less  than  6 

5  6  or  more 

6.  Years  of  organizational^termediate  level  maintenance  experience  (Include  both 
managerial  and  technical  experience.  Do  not  include  instmctional  tours) 

2  none 
_ less  than  2 

3  more  than  2,  but  less  than  4 

5  more  than  4,  but  less  than  6 

7  6  or  more 

7.  Years  of  depot  level  maintenance  experience 

15  none 

_ less  than  2 

1  more  than  2,  but  less  than  4 

1  mwe  than  4,  but  less  than  6 

_ 6  or  more 

8.  Years  since  you  filled  one  of  the  positions  in  6  or  7  above 

2  not  applicable  (have  not  filled  a  position  identified  in  questions  6  or  7) 

3  less  than  2 

6  more  than  2,  but  less  than  4 

2  more  than  4,  but  less  than  6 

4  6  or  more 
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Section  B:  Quality  Issues 


1.  BARRIERS  TO  EFFECTIVE  TRAINING:  The  first 
round  responses  indicate  that  the  following  are  barriers  to 
effective  training.  Please  rate  your  agreement/disagreement 
with  each. 

Additionally,  nlease  identify,  bv  circling  the  letters  in  the 
left  hand  colunui,  the  ten  barriers  that  you  believe  are  the 
most  significant.  [7  No  Responses] 

s 

T 

R 

O  A 
N  G 
G  R 
L  E 
Y  E 

A 

G 

R 

E 

E 

D 

I 

S 

A 

G 

R 

E 

E 

S  D 
T  I 

R  S 
O  A 
N  G 
G  R 
L  E 
Y  E 

N  R 

O  E 

S 

P 

o 

N 

S 

E 

1 

Commanders  (flight,  squadron,  group,  and  wing)  have  limited 
training  background.  [2  votes] 

1 

6 

1 

1 

1 

B 

Commanders  (flight,  squadron,  group,  and  wing)  have  limited 
maintenance  experience.  [5  votes] 

[(Dis)  SQ  CC  is  limited,  all  others  are  MX  types] 

6 

6 

1 

1 

C 

Restrictions  on  maximum  class  size  and  student  throughput 
result  in  continuous  back-to-back  classes.  [7  votes] 

10 

1 

3 

1 

1 

1 

Manning  levels  and  back-to-back  classes  afford  instructors 
inadequate  time  for  conducting  research  and  course 
development.  [10  votes] 

16 

1 

1 

1 

1 

Q 

Officer  and  enlisted  training  are  not  distinguished  from  one 
another  by  present  training  policy.  [2  votes] 

5 

5 

1 

2 

1 

n 

AMMOC's  designation  as  technical  training  precludes  general 
management  training.  [4  votes] 

5 

6 

5 

1 

1 

1 

Course  Training  Standard  (CTS)  objectives  do  not  support  an 
underlying  goal  for  AMMOC's  training.  [2  vote] 

5 

8 

3 

1 

1 

G 

AMMOC  lacks  authority  to  modify  the  CTS.  [6  votes] 

5 

8 

2 

1 

1 

H 

Process  for  changing  course  length  is  difficult.  [6  votes] 

1 

■ 

M 

1 

AMMOC  is  not  included  on  important  message  traffic 
pertaining  to  maintenance  policies  and  practices.  [5  votes] 

12 

1 

1 

1 

1 

I 


Utilization  and  Training  Workshops  (U&TW)  are  too 
infrequent.  [3  votes] 


K  Vaguely  worded  CTS  objectives  make  it  difficult  to  determine 
the  intent  of  the  CTS  authors.  [5  votes] 


AMMOC  instructor  staff  lacks  educational  expertise  in 
academics  and  training.  [3  votes] 


M  AMMOC  instructors  arc  frequently  pulled  from  the  course  to 
fill  other  positions  in  the  group.  [S  votes] 


N  There  is  no  incentive  to  streamline  training  because  any 
resources  saved  are  taken  away.  [5  votes] 

[(Dis)  I  don't  believe  there  is  an  incentive  to  change  the  course 
length,  but  I  don't  think  this  is  the  reason.  I  believe  it  is  more  to 
do  with  resistance  to  change.] 


O  Classes  must  remain  in  session,  even  after  teaching  objectives 
have  been  satisfred.  [4  votes] 


Senior  management  adds  training  requirements  to  AMMOC 
above  and  beyond  what  is  called  for  in  the  CTS  (i.e.  brake 
changes).  [2  votes] 

[(NR-  question  was  double  marked  Agree  &  Etisagree, 
interpreted  as  neutral  or  no  response)] 

[(Strg  Dis)  CTS  item  "perform  a  maintenance  task"  is  currently 
fulfilled  by  the  MOC  (Maintenance  Operations  Center) 
simulator.  This  item  ("perform  a  maintenance  task")  would  be 
fulfilled  by  a  brake  change.] 


Q  Accessibility  to  current  tecrmology  trainers  is  lacking.  [3  votes] 


6  I  5  I  1  I  1 


6  2  11 


R  There  is  insufficient  funding  fOT  student  instructional  materials.  11  6 
[6  votes] 


S  AMMOC  supervisenry  positions  are  staffed  widi  personnel  of 
insufficient  rank.  [2  votes] 


1 

Approval  for  acceleration  of  students  or  instruction  is  difficult 
to  obtain.  As  a  result,  such  measures  are  usually  not  pursued. 

[2  votes] 

6 

8 

2 

1 

1 

u 

Proficiency  advancement  test  failures  are  scored  as  block  test 
failures.  As  a  result,  proficient  students  are  hesitant  to  attempt 
proficiency  advancement.  [3  votes] 

8 

3 

6 

1 

1 

COMMENTS:  (Hease  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 
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2.  CUSTOMER  ORIENTATION:  The  first  round 
responses  identified  the  individuals  or  organizations  listed 
below  as  AMMOC's  customers.  Please  rate  your 
agreement/disagreement  with  each  as  a  customer. 


B 

AMMOC  students 

B 

Supervisors  of  AMMOC  graduates 

C 

Commanders  of  AMMOC  graduates 

D 

MAJCOMs,  Field  Operating  Agencies,  or  Direct  Reporting 

Units  to  which  AMMOC  graduates  are  assigned 

E  Foreign  governments  of  international  students 


3.  FEEDBACK:  According  to  AETCR  52-12,  the 
following  are  means  of  obtaining  feedback.  Please  indicate 
vour  ooinion  of  how  effective  the  oresent  uses  of  these 
measures  are  for  providing  AMMOC  with  meaningful 
feedback. 

H  E 

I  F 
G  F 
H  E 
L  C 
Y  T 

I 

V 

E 

I 

N 

E 

F 

F 

E 

C 

T 

I 

V 

E 

H  I 

I  N 
G  E 
H  F 
L  F 
Y  E 
C 
T 

I 

V 

E 

1 

Training  (^ality  Reports  (TQRs). 

[(NR)  Never  seen  these.  Who  gets  them?] 

[(Highly  Ineff)  Never  seen  one  come  back] 

2 

8 

3 

B 

Field  Evaluation  Quesdonnaiies/Training  Evaluation  Reports 
(TERs). 

[(NR)  Never  seen  these.  Who  gets  them?] 

1 

6 

3 

C 

Customer  Service  Information  Line  (CSIL). 

■ 

3 

6 

5 

3 

Field  Interviews. 

[(Agree)  Just  started] 

3 

6 

1 

1 

1 

E 

Occupational  Survey  Reports 

[(Highly  Ineff)  Worthless] 

[(Highly  Ineff)  Stupid/a  waste  -  In  left  field  (Respondent 
indicated  they  had  seen  data  from  the  most  recent  survey)] 

1 

2 

1 

5 

2 

Subject  Matter  Expert  Feedback 

[(NR)  Unknown  to  me] 

1 

6 

6 

2 

3 

G 

Graduate  Assessment  Surveys. 

[(NR)  Unknown  to  me] 

1 

8 

5 

3 

1 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 


[3.A-G.  Instructors  here  (at  AMMOC)  have  never  seen  any  of  these!] 


G-8 


4.  COMMUNICATIONS:  The  first  round  questionnaire 
demonstrated  that  the  majority  of  AMMOC  instructors 
regard  communications  with  Air  Staff,  the  major 
commands,  and  the  field  as  relatively  ineffective.  Please 
rate  your  agreement/disagreement  with  each  of  the 
following  improvement  ideas  submitted  during  the  first 
round. 

s 

T 

R 

O  A 
N  G 
G  R 
L  E 
Y  E 

A 

G 

R 

E 

E 

D 

I 

S 

A 

G 

R 

E 

E 

S  D 
T  1 

R  S 
O  A 
N  G 
G  R 
L  E 
Y  E 

N  R 
0  E 

S 

P 

0 

N 

S 

E 

1 

Provide  AMMOC  access  to  electronic-mail  for  correspondence 
with  organizations  throughout  the  Air  Force. 

8 

8 

1 

1 

B 

Permit  and  fund  instructor  TDYs  to  operational  wings  to 
maintain  AMMOC's  currency  with  field  operations  and  policies. 

13 

1 

1 

1 

C 

Place  AMMOC  on  distribution  for  changes  in  policies  and 
regulations. 

14 

3 

1 

1 

Include  AMMOC  in  pertinent  message  traffic  for  informational 
purposes  to  maintain  AMMOC's  currency  on  unit  level 
maintenance  policies  and  practices. 

13 

3 

1 

1 

E 

Establish  points  of  contact  within  MAJCOMs,  research  centers 
and  depots.  These  points  of  contact  could  forward  information 
to  AMMOC. 

11 

5 

1 

1 

Have  instructors  aggressively  follow-up  with  operational  units 
to  find  out  how  the  students'  time  at  AMMOC  prepared  them 
for  their  assignments. 

6 

1 

3 

1 

G 

Have  instructors  conduct  field  evaluations  of  AMMOC 
graduates. 

6 

1 

3 

1 

H 

Conduct  surveys  of  former  students  and  commanders. 

6 

9 

2 

■ 

■ 

I 

Provide  a  system  through  which  the  various  orgaiuzations  may 
provide  inputs  or  suggestions  pertaining  to  AMMOC's  training 
on  an  ongoing  basis,  not  only  at  a  U&TW  conference. 

10 

5 

2 

1 

1 

1 

Implement  a  guest  speaker  program  featuring  speakers  with  a 
variety  of  backgrounds.  This  will  expose  instructors  to  current 
policies  and  practices. 

8 

5 

2 

1 

1 
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5.  ADAPTABILITY  TO  CHANGE:  The  first  round  questionnaire  demonstrated  that 
the  majority  of  AMMOC  instructors  regard  the  present  system  of  training  development  as 
relatively  inflexible  and  nonresponsive.  The  following  two  sections  address  opportunities 
fOT  improving  the  responsiveness  and  flexibility  of  training. 


a.  During  round  one,  AMMOC  instructors  suggested  the 
following  measures  for  improving  the  flexibility  [*]  of 
training.  Please  rate  your  level  of  agreement/disagreement 
with  each. 

[*  "flexibility"  should  have  read  "responsiveness."  See  page 
4-12  of  the  thesis  for  explanation.] 
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B 

Allow  instructors  to  go  TDY  and  then  update  lesson  plans. 

12 

B 

1 

■ 

1 

B 

Provide  instructors  time  to  conduct  course  development. 

[(Strg  Agree)  We  need  to  do  either  B  or  C.] 

15 

2 

1 

1 

1 

C 

Establish  a  cadre  of  personnel  for  course  development  to  relieve 
instructors  of  the  burden  of  both  developing  and  providing 
training. 

[(Strg  Agree)  We  need  to  do  either  B  or  C.] 

8 

2 

1 

1 

Shorten  instructor  tour  lengths  to  maintain  an  instructor  staff 
with  recent  experience. 

[(Dis)  3  yrs  is  O.K.-if  you  can  keep  them  in  AMMOC] 

[(Dis)  Shorter  tour  lengths  would  require  a  constant  flow  of 
getting  people  qualified  and  then  only  getting  a  year  or  so  of 
work  before  going  through  the  same  process  all  over  again.  It 
takes  8-10  months  to  get  an  instructor  on  their  own  in  the 
classroom.] 

6 

6 

1 

E 

Distinguish  course  development  policies  for  officer  courses 
from  those  for  enlisted  courses. 

[(Strg  Dis)  What?  We  use  (enlisted  course)  CDCs  to  develop 
our  courses.  Why  is  this  question  asked?] 

1 

9 

_ 

2 

1 

_ 1 

1 
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1 

Establish  MAJCOM  and  Air  Staff  points  of  contact  for 
AMMOC  which  include  office  symbols,  names  and  telephone 
numbers. 

1 

8 

2 

1 

G 

Empower  AMMOC  staff  to  change  the  Course  Training 
StaiKlard  (CTS)  when  changes  in  maintenance  policies  and 
practices  warrant  it. 

6 

8 

2 

1 

H 

Send  MAJCOM  and/or  Air  Staff  to  AMMOC  periodically  to 
evaluate  appropriateness  of  its  curriculum. 

1 

1 

6 

1 

I 

Include  AMMOC  in  pertinent  message  traffic  for  informational 
purposes  to  maintain  AMMOC's  currency  on  unit  level 
maintenance  policies  and  practices. 

11 

6 

1 

1 

Improve  communication  between  AMMOC,  and  USAF  and 
MAJCOMs  regarding  impending  changes  to  maintenance 
policies  and  practices. 

11 

6 

1 

1 
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b.  During  round  one,  AMMOC  instructors  suggested  the 
following  as  measures  for  imorovine  the  flexibilitv  of 
training.  Please  rate  your  level  of  agreement/disagreement 
with  each. 
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1 

Implement  mission-oriented  management  philosophies  which 
embrace  principles  such  as  empowerment  and  decentralization. 

5 

10 

1 

1 

B 

Send  instructors  TDY  to  operational  units  to  increase  their 
awareness  of  current  policies  and  practices. 

11 

6 

1 

1 

C 

Distinguish  course  development  responsibilities  between 
AMM(Xr  instructors  and  Training  Development  Element 
(TDE)  personnel. 

8 

1 

1 

1 

Give  instructors  inputs  on  planning  course  schedules,  class 
sizes,  and  down  times. 

12 

5 

1 

1 

E 

Include  AMMOC  in  pertinent  message  traffic  for  informational 
purposes  to  maintain  AMMOCs  currency  on  unit  level 
maintenance  policies  and  practices. 

12 

1 

1 

1 

COMMENTS;  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  l.A.) 


[5.b.B  &  D  (Stng  Agree)  We  have  a  lot  of  outstanding,  common  sense  maintenance 
officers  here  who  want  to  do  a  great  job  but  have  their  hands  ded.  Allowing  us  to  keep 
current  with  field  operating  (5.b.B)  and  the  time  to  develop  quality  courses  (5.b.D)  will 
only  improve  this  course  overall.  The  credibility  of  instructors  is  challenged  when  we  brief 
ixrlicies  that  are  no  longer  in  effect  or  talk  about  how  great  changes  are  working  when  in 
reality  it  is  all  garbage.  Communications  between  the  field  both  in  person  and  written  is 
the  key.] 


G-12 


% 


6.  INSTRUCTOR  PERFORMANCE  EVALUATION: 

The  responses  from  the  first  round  denumstrated  that 
instructor  performance  is  evaluated  primarily  with  the 
AETC  Form  281,  in  addition  to  some  other  informal  means 
such  as  student  performance.  During  the  first  round,  the 
following  ideas  were  suggested  for  improving  the  methods 
for  evaluating  instructor  performance.  I^ease  rate  your 
agreement/disagreement  with  these  ideas. 
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1 

AETC  Form  281  should  place  more  emphasis  on  lesson  content 
than  instructor  presentations  skills. 

2 

5 

8 

1 

1 

B 

Implement  a  more  descriptive  AETC  Form  281  giving  more 
specific  expectations  of  performance. 

3 

11 

3 

1 

1 

C 

Ensure  instructors  are  evaluated  by  personnel  who  possess  both 
subject  matter  and  training/education  expertise. 

[(Dis)  Impossible] 

6 

1 

1 

Conduct  evaluations  more  frequently  to  obtain  a  more 
representative  measurement  of  instructor  performance. 

1 

8 

6 

2 

1 

E 

Have  a  variety  of  experts  evaluate  each  instructor  to  improve 
quality  of  feedback. 

1 

1 

5 

1 

1 

1 

Conduct  informal  evaluations  in  addition  to  formal  evaluations. 

3 

11 

2 

■ 

1 

G 

If  student  test  scores  are  to  serve  as  a  measure  of  instructor 
performance  use  random  computer  generated  tests  to  prevent 
"teaching  the  test." 

IfAerce)  Grades  should  not  be  used  as  an  instructor's 
performance  measurement  tool.] 

3 

9 

1 

H 

Base  instructor  performance  on  student  performance  as 
measured  by  randomly  generated  pretests  and  post  tests. 

[(Dis)  Yes  -  Random  test  generation  is  a  good  idea.] 

3 

6 

6 

1 

1 
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1 

Increase  student  involvement  in  instructor  performance 
evaluations. 

2 

1 

6 

5 

1 

1 

[(Dis)  How?] 

1 

1 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 

[A  good  assessment  program  must  include  1 .  Supervisor  Assessment 

2.  Peer  Assessment 

3.  Seif  Assessment  ] 
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7.  AMMOC  PERFORMANCE  EVALUATION:  From  the 
first  round  responses,  it  is  evident  that  there  is  little 
awareness  of  any  systematic  method  for  evaluating 
AMMOC's  overall  performance.  AMMOC  instructors 
offered  the  following  as  methods  for  improving  AMMOC 
performance  measures. 

s 

T 

R 

0  A 
N  G 
G  R 
L  E 
Y  E 

A 

G 

R 

E 

E 

D 

I 

S 

A 

G 

R 

E 

E 

1^ 

N  R 
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Get  an  evaluation  "expert"  to  develop  an  evaluation  plan. 

[(Strg  Agree)  We  have  the  "experts"  on  the  staff,  we  just  need 
to  have  leadership  recognize  &  embrace  that  expertise.] 

3 

11 

1 

1 

1 

B 

Administer  a  pretest  and  cumulative  final  exam  at  the  end  of  the 
course  as  a  measure  of  effecdveness. 

[(Dis)  Block  9  serves  as  a  cumulative  test  if  administered 
properly.] 

8 

1 

1 

Improve  field  surveys  to  provide  more  useful  information  to 
AMMOC. 

5 

10 

2 

1 

1 

1 

Have  maintenance  officers,  as  opposed  to  personnel  with  no 
background  in  aircraft  maintenance,  conduct  field  surveys  of 
AMMOC  graduates. 

1 

8 

1 

1 

1 

E 

Aimual  questionnaire  to  field  commanders  about  how  our 
students  are  doing  and  what  we  can  do  to  help  them  out. 

1 

8 

1 

1 

1 

1 

Evaluate  graduates'  job  performance  in  the  field. 

[(Strg  Dis)  Eval.  knowledge] 

5 

9 

2 

1 

1 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  l.A.) 

[7.C  (Dis)  &  7.D  (Agree)  These  survey  rarely  work.  They  are  more  a  nuisance  to 
supervisors  and  commanders  in  the  field  than  anything  else.  This  is  just  square  filling  and 
offers  little  good  information.] 
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8.  CRITIQUE  PROGRAM:  During  round  one,  the 
following  were  suggested  as  means  of  improving  the 
critique  program.  I^ease  rate  your  agreement/ 
disagreement  with  each. 


1 

Limit  review  process  of  student  critiques  to  the  instructor 
supervisor  level. 

[(Dis)  Limit  to  Flight  Commander  level] 

B 

Establish  a  requirement  for  AMMOC  to  reply  to  all  student 
critiques  regardless  of  whether  students  so  request. 

C 

Do  not  make  student  critiques  mandatory. 

1 

Have  students  discuss  their  critiques  with  each  instructor  prior 
to  graduation. 

E 

Conduct  group  reviews  of  student  critiques  with  AMMOC 
instructor  staff  to  help  determine  core  problems  and  devise 
solutions. 

Change  attitude  of  management  to  listen  and  be  receptive  to 
new  ideas,  even  if  they  come  from  students. 

[(Agree)  We  do  listen] 

[(Strg  Dis)  Fit  level  ragt  attitude  excellent  &  receptive] 

G 

Corrective  actions  to  deficiencies  noted  in  student  critiques 
should  be  coordinated  with  the  instructor  staff. 

H 

Follow  up  on  corrective  actions  implemented  as  result  of 
student  critiques. 

I 

Ensure  end  of  course  critiques  are  provided  to  instructors  being 
critiqued. 
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H 

Delay  course  critiques  for  a  period  of  one  year  after  graduation. 

2 

3 

8 

i 

■ 

[(Agree)  Do  both] 

[(Agree)  One  year  might  be  a  little  long.  6-8  months  probably 
better.] 

[(Dis)  Qitique  at  end  of  course  plus  one  year  after  grad. 
-Sometimes  students  don’t  realize  the  benefit  of  AMMOC  until 
they  have  been  out  there  a  while!] 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 
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9.  COURSE  OBJECTIVE  DEVELOPMENT 


a.  During  round  one,  AMMOC  instructors  identified  the 
folloiring  as  strengths  and  weaknesses  with  the  course 
objective  development  process.  Please  rate  your  agreement 
or  disagreement  with  those  presented  below. 
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Course  objectives  are  very  precise,  orderly,  and  clearly  defined.  1  8 


[(Strg  Dis)  Our  objectives  are  extremely  vague  and  the  process 
of  changing  the  course  is  very  cumbersome.] 


B  Once  changes  are  made  to  the  objectives,  the  course  responds  1  5  10  1 

with  course  updates  relatively  quickly. 

[(Dis)  Our  objectives  are  extremely  vague  and  the  process  of 
changing  the  course  is  very  cumbersome.] 


C  There  are  no  real  management/performance  objectives.  Our  10  5  2 

course  objectives  only  require  students  to  iden^y  facts.  This  is 
the  lowest  level  of  the  cognitive  domain. 


Objectives  are  developed  at  U&TW  meetings  by  senior 
maintenance  personnel  from  the  MAJCOMs  and  Air  Staff,  not 
maintenance  officers  from  the  field. 

[(Strg  Dis)  We  develop  objectives] 

[(Strg  Dis)  Objectives  are  not  developed  at  U&TW,  only 
behavior  statements] 

[(Dis)  disagree  as  a  weakness,  not  the  statement  itself.] 


E  Old  OTS  is  often  used  as  a  building  block  for  new  CTS.  This 
methodology  does  not  lend  itself  to  significant  changes. 


Course  objectives  devised  from  U&TW  are  vague.  For 
example,  an  objective  that  reads  "Identify  facts  about 
electricity"  could  be  interpreted  numerous  ways. 

[(NR)  Objectives  are  not  developed  at  U&TW,  only  behavior 
statements] 


G 

Course  objectives  rarely  change. 

6 

8 

3 

■ 

■ 

H 

It  is  difficult  to  determine  the  intentions  of  the  CTS  writers. 

For  example,  a  CTS  element  may  say,  "Identify  facts  about..." 
What  kind  of  facts  do  they  want  taught? 

11 

1 

2 

1 

I 

CTS  development  lags  changes  in  the  field. 

12 

i 

1 

■ 

■ 

COMMENTS:  (Please  refer  to  the  item  to  which  you  arc  commenting;  for  example,  LA.) 


G-19 


b.  During  round  one,  AMMOC  instructors  suggested  the 
following  as  improvements  to  the  course  objective 
development  process.  Please  rate  your  agreement  or 
disagreement  with  each  of  them. 
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Develop  an  "on-line"  system  for  developing  and  maintaining 
course  objectives  that  is  accessible  by  AMMOC,  Air  Staff,  and 
the  major  commands. 

((Strg  Agree)  (Respondent  lined  out  objectives  and  wrote 
goals)] 

8 

1 

1 

B 

AMMOC  should  be  empowered  to  develop  its  own  course 
objectives  with  the  oversight  of  MAJCOMs  and  Air  Staff. 

9 

5 

3 

1 

1 

C 

AMMOC  should  be  empowered  to  develop  course  objectives, 
and  represent  the  various  MAJCOMs  in  the  process. 

1 

6 

5 

1 

1 

Course  objectives  should  support  an  overall  plan  for 
maintenance  officer  training,  and  should  not  exist  as  a  list  of 
independent  subjects. 

9 

1 

1 

1 

1 

E 

Course  objectives  should  be  developed  with  the  involvement  of 
maintenance  leaders  from  operational  units  and  MAJCOM 
staffs  who  range  from  company  to  field-grade  ranks.  The 
development  process  should  include  a  review  of  current 
AMMOC  lesson  plans. 

9 

5 

3 

_ 

1 

1 

Each  CTS  element  should  have  a  short  narrative  statement 
outlining  what  aspects  of  the  topic  should  be  taught.  This  will 
convey  the  authors'  intents  pertaining  to  each  CTS  element 

9 

1 

1 

1 

1 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 


[9.b.E  &  9.b.F  These  are  the  solutions  to  AMMOC.  I've  spoken  to  LGs  who  disagree 
with  material  taught  in  AMMOC.  They  have  not  ever  been  included  in  (occupational 
measurement)  surveys.  Why?  Overhaul  neededi 
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10.  COURSE  CONTROL  DOCUMENTS  (Course  Training  Standards,  Course 
Training  Charts,  and  Plans  of  Instruction). 


a.  During  round  one,  AMMOC  instructors  identified  the 
following  as  strengths  and  weaknesses  of  course  control 
documents.  Please  rate  your  agreement  or  disagreement 
with  those  presented  below. 
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Stireiij^hsx"'' 

1 

Course  training  charts  provide  a  fast  overview  of  the  course  and 
total  training  time. 

6 

10 

1 

1 

1 

B 

Course  charts  show  the  flow  of  the  overall  course  from  one 
block  to  the  next. 

2 

12 

2 

1 

1 

' 

1 _ 

The  POI  is  the  most  useful  document  for  the  instructor  because 
it  provides  the  most  insight  into  the  training  itself. 

1 

9 

6 

2 

1 

1 

Course  control  document  formats  serve  the  purpose  of 
standardizing  training. 

1 

14 

2 

1 

1 

■ 

Weaknesses 

E 

There  are  too  many  course  control  documents. 

■ 

6 

10 

■ 

■ 

1 

Course  control  documents  quickly  become  outdated. 

6 

10 

1 

■ 

■ 

G 

Course  control  documents  require  too  much  time  to  keep 
current. 

2 

8 

1 

1 

2 

H 

CTS,  course  charts  and  POI  pan  1  do  not  provide  useful 
guidance. 

3 

6 

1 

1 

1 

I 

Obtaining  permission  to  modify  course  charts  and  POI  part  1  is 
difficult 

[(Strg  Dis)  Our  TDB  is  very  efficient  in  our  needs] 

5 

9 

2 

1 

1 

1 

Lesson  plans  (POIs  part  2)  are  only  as  good  as  the  responsible 
instructor  makes  them;  they  are  the  sole  users  in  most  cases. 

11 

5 

1 

1 

1 
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K 

POIs  are  full  of  "facts"  and  "technical"  information  obtained 
from  regulations  and  technical  orders  which  have  minimal 
relevance  to  the  development  of  managers. 

7 

4 

4 

1 

1 

[(Strg  Dis)  True,  they  contain  a  lot  of  T.O.  and  regulation  info 
but  a  primary  job  of  managers  is  to  enforce  compliance.  How 
can  you  do  that  if  you  do  not  know  the  standards?  On  the  other 
hand,  AMMOC  should  teach  more  "real-life"  management 
scenarios.  Spend  about  an  hour  a  day  in  class  throwing  out 
different  scenarios  dealing  with  people  or  planning  etc.  and  let 
the  students  discuss  them.  The  majority  of  the  (students)  do 
not  even  realize  they  will  encounter  situations  like  these  in  the 
field.  In  this  respect,  we  are  doing  our  students  a  disservice  by 
giving  them  aU  sorts  of  technical  info  and  no  ways  to  apply  it  to 
real  situations.  90%  of  our  job  is  people!!] 

1 

POIs  are  rarely  reviewed  for  appropriateness  and  applicability 
to  mission  goals. 

5 

1 

5 

1 

1 

M 

The  instructional  guidance  for  lesson  plans  is  a  product  of  the 
instructor's  "interpretation"  of  CTS  elements  rather  than  a 
concrete  statement  of  training  desires. 

1 

5 

1 

1 

1 

COMMENTS;  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  l.A.) 
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b.  During  round  one,  AMMOC  instructors  suggested  the 
following  as  improvements  to  course  control  documents. 
Please  rate  your  agreement  or  disagreement  with  those 
presented  below. 
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1 

Condense  factual  information  in  POIs  into  workbooks  that  can 
be  completed  at  home  base  with  no  loss  of  quality. 

3 

1 

1 

1 

2 

B 

Streamline  course  control  documents. 

[(Strg  Agree-iespondent  inserted  "the  process  for  developing" 
after  "Streamline")] 

3 

9 

1 

C 

Allow  AMMOC  to  modify  course  control  documents. 

5 

8 

i 

■ 

■ 

1 

Scrap  the  current  CTS  and  start  from  scratch~do  not  try  to 
"mend"  the  current  CTS. 

8 

6 

3 

1 

1 

E 

Give  instructors  the  time  or  AMMOC  the  manning  to  properly 
develop  the  course  chart  and  POIs. 

13 

1 

1 

1 

1 

1 

Make  objectives  clear  and  concise. 

11 

6 

■ 

■ 

■ 

Make  it  easier  for  insuxictors  to  rearrange  objectives  within  a 
block  and  even  move  objectives  from  one  block  to  another  if  it 
would  help  improve  training. 

[(Agree)  So  far  our  supervision  has  allowed  movement  of 
objectives  without  interference.] 

[(Dis)  Has  been  done,  can  be  done  easy.] 

[(Strg  Agree)  It's  easy  to  do!] 

[(Agree)  I  feel  that  TDB  does  this  already  when  necessary] 

10 

6 

1 

COMMENTS:  (Please  refer  to  the  item  to  which  you  are  commenting;  for  example,  1  .A.) 
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11.  GENERAL  ISSUES:  Presented  below  are  general 
ideas  su^ested  during  the  flrst  round.  Please  rate  your 
agreement/disagreement  with  each. 


1 

Maintenance  officers  should  be  offered  an  incentive  to  become 
AMMOC  instructors  (i.e.  choice  of  follow-on  assignment). 

1 

AMMOCs  name  should  be  changed  to  establish  a  new 
reputation. 

c 

A  logistics  university  should  be  created  that  encompasses  all 
support  officer  career  fields.  This  university  would  focus  not 
only  on  teaching,  but  also  on  research  and  development  of 
logistics  processes  and  management  techniques. 

i 

Facilities  should  be  improved. 

E 

Officer  training  should  be  separated  fiom  enlisted  training.  This 
would  prevent  AMMOC  students  from  being  treated  like 
"airman  basics"  by  the  squadron  and  other  base  agencies. 

[(Strg  Dis)  Use  more  enlisted  courses  and  enlisted  MSgt  & 
SMSgts  to  teach  AMMOC.  (If  well  get  more  technical)] 

AMMOC  should  not  be  designated  as  technical  training. 

[(Agree)  AMMOC  is  not  a  technical  training  course] 

G 

AMMOC  curriculum  should  be  refocused  to  emphasize 
management  (i.e.  problem  solving,  decision  making,  etc.). 
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H  Make  AMMOC  a  seminar  course  without  block  tests. 
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i 


[(Sti^  Dis)  I  do  not  think  this  is  a  good  idea  because  the 
younger  people  we  work  with  are  not  always  dedicated  enough 
to  perform  in  a  seminar  like  the  senior  officers  course.  Test  are 
still  a  good  idea  to  have.] 

[(Dis)  There  are  areas  we  could  teach  by  seminar,  however,  not 
M  areas  should  be.  Some  traditional  instruction,  individual 
projects,  seminars/guest  speakers,  simulators/exercises,  hands- 
on,  and  tours  would  round  out  AMMOC  well.] 

[(Strg  Dis)  Seminar  with  block  test] 


Add  authorizatitms  for  select  personnel  with  advanced  academic 
degrees  in  education  and  training  disciplines. 

[(Dis)  What  we  need  is  fewer  homesteaders  and  more  people 
with  more  than  one  assignment  in  the  field  to  come  here  and 
teach  real-life.  Advanced  degrees  do  not  make  experts  they 
make  people  who  know  the  principles  (but)  have  very  little 
experience  in  applying  them.] 

[(Dis)  (We  have)  open  (authorizations)] 


Establish  library/research  area  fw  students  and  staff. 


[(Dis)  Give  (us)  time  (for  research)] 


K  Establish  authorizations  within  AMMOC  for  a  select  number  of  5 
higher  ranking  officers  (i.e.  majors/lieutenant  colonels)  for 
supervisory  and  instructor  duty. 
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[(Dis)  Give  (us)  more  time/material] 

[(Strg  Agree)  Not  just  higher  rank,  but  people  with  more 
experience  than  1  base,  1  aircraft  (Someone  with  state  side  and 
overseas  time)  so  they  understand  the  different  areas.  Make 
AMMCX^  Staff  positions.] 


Institute  job  related  exercises  pertaining  to  topics  such  as 
mobility  and  battle  staffs. 


M  Utilize  officers  assigned  to  AMMOC  as  instructors  or  course 
developers  instead  of  employing  them  in  other  positions  in  the 
group. 
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N 

Formally  include  AMMOC  instructors  in  the  feedback  process 
for  training  policy  and  course  development. 

8 

9 

1 

1 

1 

0 

Bring  AMMOC  more  in  line  with  Quality  Air  Force  (QAF) 
practices. 

1 

1 

1 

1 

COMMENTS;  (Please  refer  to  the  item  to  which  you  ate  commenting;  for  example,  I.A.) 
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H-1 
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Equations 

a 

(1) 

n 


Where,  Rsnumber  of  reqxxises,  and  x=value  assigned  to  each  response  (strongly 
agree=4,  agree=3,  disagree=2,  strongly  disagree=l). 
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